Cited by 1,072Mobile Crowdsensing and CrowdsourcingSurvey Methodology and NonresponseEthics in Clinical Research

The DOI matches its Crossref record; 174 of the 177 references it cites were found in a registry or an index.

Conducting Clinical Research Using Crowdsourced Convenience Samples

Chandler, Jesse, Shapiro, Danielle·2016·Annual Review of Clinical Psychology

DOI 10.1146/annurev-clinpsy-021815-093623

Open this paper in CoChat — free account

Abstract

Crowdsourcing has had a dramatic impact on the speed and scale at which scientific research can be conducted. Clinical scientists have particularly benefited from readily available research study participants and streamlined recruiting and payment systems afforded by Amazon Mechanical Turk (MTurk), a popular labor market for crowdsourcing workers. MTurk has been used in this capacity for more than five years. The popularity and novelty of the platform have spurred numerous methodological investigations, making it the most studied nonprobability sample available to researchers. This article summarizes what is known about MTurk sample composition and data quality with an emphasis on findings relevant to clinical psychological research. It then addresses methodological issues with using MTurk--many of which are common to other nonprobability samples but unfamiliar to clinical science researchers--and suggests concrete steps to avoid these issues or minimize their impact.

Integrity signals

Registry

Crossref · 10.1146/annurev-clinpsy-021815-093623

Cites

174 of 174 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Self-citation
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Self-citation
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Ed, H., & Suh, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,941

Show all 177 references

Research works printed but not linked: 12

Barnhoorn JS, 2014, Behav. Res. Methods, 530, 1
Beramendi P, Duch RM, Matsuo A. 2014. When lab subjects meet real people: comparing different modes of experiments. Presented at Asian Polit. Methodol. Conf., Taipai
Casey L, Chandler J, Levine AS, Proctor A, Strolovitch D. 2015. Demographic characteristics of a large sample of US workers. Mathematica Policy Res. Rep. In press
Downes-Le Guin T, 2006, ESOMAR World Res. Conf. Panel Res.
Higgins C, 2010, Proc. NAACL HLT 2010 Workshop on Creat. Speech and Language Data with Amazon's Mechanical Turk, Los Angeles, 89
10.1007/s10683-011-9273-9 · DOI
10.1145/2676431.2676437 · DOI
Lang ASID, 2011, Code4Lib J., 15, 1
Lebreton P, 2015, Proc. SPIE Conf. Hum. Vis. Electron. Imaging, 20th, San Francisco
10.1026//1618-3169.49.4.243 · DOI
Shao W, 2015, Digit. Cult. Educ., 7
Suri S, 2011, AAAI Workshops, 25th, San Francisco, 61

Sorted by citations

Items 601–700 of 1,001

Factor Structure and Further Validation of the 20-Item Short Form of the Obsessive Beliefs Questionnaire
Fergus, T. A., Latendresse, S. J., & Wu, K. D. (2017) · Assessment · doi:10.1177/1073191117745988 · Cited by 9
Maladaptive variants of adaptive traits and bloated specific factors
Oltmanns, J. R., & Widiger, T. A. (2018) · Journal of Research in Personality · doi:10.1016/j.jrp.2018.08.006 · Cited by 9
Patients' Perceptions of Different Information Exchange Mechanisms: An Exploratory Study in the United States
Esmaeilzadeh, P. (2020) · Methods of Information in Medicine · doi:10.1055/s-0040-1721784 · Cited by 9
The Relationship between Social Norms, Avoidance, Future Orientation, and Willingness to Engage in Climate Change Advocacy Communications
Latkin, C. A., Dayton, L., Coyle, C. R., Yi, G. T., Lee, D., & Winiker, A. K. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182413037 · Cited by 9
For Better or Worse: Self-reported Changes in Kratom and Other Substance Use as a Result of the COVID-19 Pandemic
Rogers, J. M., Smith, K. E., Schriefer, D., & Epstein, D. H. (2022) · Substance Abuse Research and Treatment · doi:10.1177/11782218221123977 · Cited by 9
Probability Discounting and Cardiovascular Risk: The Effect of Side-Effect Severity and Framing
Asgarova, R., Macaskill, A. C., ROBINSON, B. J., & Hunt, M. J. (2017) · The Psychological Record · doi:10.1007/s40732-017-0243-2 · Cited by 9
“It Only Tells Me How I Slept, Not How to Fix It”: Exploring Sleep Behaviors and Opportunities for Sleep Technology
Zhang, S., Schaub, F., Feng, Y., & Sadeh, N. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-15742-5_71 · Cited by 9
Digital signatures: a tool to prevent and predict dishonesty?
Koning, L., Junger, M., & van Hoof, J. J. (2020) · Mind & Society · doi:10.1007/s11299-020-00237-1 · Cited by 10
Applications of operant demand to treatment selection II : Covariance of evidence strength and treatment consumption
Gilroy, S. P., & Feck, C. (2022) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.735 · Cited by 9
Applications of operant demand to treatment selection I: Characterizing demand for evidence‐based practices
Gilroy, S. P., Waits, J. A., & Kaplan, B. A. (2022) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.731 · Cited by 9
Narcissus’ belief about his body: Aspects of narcissism, body image, and eating disorder symptoms
Szymczak, P., Talbot, D., Gritti, E. S., & Jonason, P. K. (2023) · PLoS ONE · doi:10.1371/journal.pone.0293578 · Cited by 8
Does Failure Help or Harm? Linking Parents’ Treatment Histories, Views of Failure, and Expectancies for Child Psychotherapy
Garibaldi, P. M., Abel, M. R., Snow, R. L., & Schleider, J. L. (2019) · Child & Youth Care Forum · doi:10.1007/s10566-019-09523-7 · Cited by 9
Crowdsourced analysis of factors and misconceptions associated with parental willingness to donate their child's organs
Jones, A., Jacobs, M. B., & October, T. W. (2019) · Pediatric Transplantation · doi:10.1111/petr.13606 · Cited by 8
Public Opinion and Expectations of Stem Cell Therapies in Orthopaedics
Puzzitiello, R. N., Dubin, J. A., Menendez, M. E., Moverman, M. A., Pagani, N. R., Drager, J., & Salzler, M. J. (2021) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2021.05.058 · Cited by 8
Research participants recruited using online labor markets may feign medical conditions and overreport symptoms: Caveat emptor
Ysidron, D. W., France, C. R., Yang, Y., & Mischkowski, D. (2022) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2022.110948 · Cited by 8
Evaluation of Health Literacy in Plastic Surgery Using a Crowdsourced Patient Survey
Bustos, V. P., Haddad, A. G., Kinney, J. R., Xun, H., Nassar, A. H., Lee, T. T. H., Lin, S. J., & Lee, B. T. (2023) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000004803 · Cited by 8
Using Reddit as a recruitment strategy for addiction science research
Goodwin, S. R., Dwyer, M. J., Caliva, S., Burrows, C. A., & Raiff, B. R. (2023) · Journal of Substance Use and Addiction Treatment · doi:10.1016/j.josat.2023.209011 · Cited by 8
Disordered Eating and Body Image Concerns in Young Adult Women With Scoliosis
Limbers, C. A., Baskin, A., & Cohen, L. A. (2023) · Clinical Medicine Insights Arthritis and Musculoskeletal Disorders · doi:10.1177/11795441231166010 · Cited by 8
Psychometric evaluation of the multiscale dissociation inventory (MDI): A replication and extension in two trauma-exposed samples
Jeffirs, S. M., Petri, J. M., Camden, A. A., Jackson, B. N., & Weathers, F. W. (2023) · European Journal of Trauma & Dissociation · doi:10.1016/j.ejtd.2023.100324 · Cited by 8
Can Food Safety Practices and Knowledge of Raw Fish Promote Perception of Infection Risk and Safe Consumption Behavior Intentions Related to the Zoonotic Parasite Anisakis?
Cancellieri, U. G., Amicone, G., Cicero, L., Milani, A., Mosca, O., Palomba, M., Mattiucci, S., & Bonaiuto, M. (2023) · Sustainability · doi:10.3390/su15097383 · Cited by 8
Longitudinal and experimental investigations of implicit happiness and explicit fear of happiness
Collins, A. C., Jordan, D. G., Bartoszek, G., Kilgore, J., Lass, A. N. S., & Winer, E. S. (2023) · Cognition & Emotion · doi:10.1080/02699931.2023.2223907 · Cited by 8
Recovery after a suicidal episode: Developing and validating the Recovery Evaluation and Suicide Support Tool (RESST).
Sokol, Y., Glatt, S., Levin, C., Tran, P., Rosensweig, C., Silver, C., Hubner, S., Primavera, L. H., & Goodman, M. S. (2023) · Psychological Assessment · doi:10.1037/pas0001269 · Cited by 8
Roles of bilingualism and musicianship in resisting semantic or prosodic interference while recognizing emotion in sentences
Neumann, C., Sares, A. G., Chelini, E., & Deroche, M. L. D. (2023) · Bilingualism Language and Cognition · doi:10.1017/s1366728923000573 · Cited by 8
Influences on sexting in an intimate relationship: Motivations, risks, communication, personality traits, and relationship variables
Falconer, T. A., Humphreys, T. P., O’Hagan, F. T., & Johnson, J. (2023) · The Canadian Journal of Human Sexuality · doi:10.3138/cjhs.2022-0052 · Cited by 8
Measurement invariance of brief forms of the Schizotypal Personality Questionnaire across convenience versus random samples
O’Hare, K., & Linscott, R. J. (2023) · Schizophrenia Research · doi:10.1016/j.schres.2023.10.033 · Cited by 9
Patient Barriers to Discontinuing Long-Acting Reversible Contraception
Fox, K. R., Kameoka, A. M., Raidoo, S., Soon, R. A., Tschann, M., Fontanilla, T. M., & Kaneshiro, B. (2023) · Maternal and Child Health Journal · doi:10.1007/s10995-023-03878-z · Cited by 8
Suicidal behavior and deliberate self-harm: A major challenge for youth residential care in Spain
Muela, A., García‐Ormaza, J., & Sansinenea, E. (2024) · Children and Youth Services Review · doi:10.1016/j.childyouth.2024.107465 · Cited by 11
Measuring delay discounting in a crowdsourced sample: An exploratory study
Rotella, A., Fogg, C., Mishra, S., & Barclay, P. (2019) · Scandinavian Journal of Psychology · doi:10.1111/sjop.12583 · Cited by 8
Perceptions of child vulnerability in first-time mothers who conceived using assisted reproductive technology
Egan, K. N., Summers, E., & Limbers, C. A. (2021) · Journal of Reproductive and Infant Psychology · doi:10.1080/02646838.2021.1896689 · Cited by 8
Offering small tangible rewards within social casino games increases in-play bets but does not impact real-money gambling
Kim, H. S. A., Rockloff, M., Leslie, R. D., McGrath, D. S., Wohl, M. J. A., & Hodgins, D. C. (2021) · Addictive Behaviors · doi:10.1016/j.addbeh.2021.106984 · Cited by 8
The Moderating Effect of Attentional Control on the Relationship Between COVID Stress and Generalized Anxiety Symptoms
Bardeen, J. R., Gorday, J. Y., & Clauss, K. E. (2021) · Psychological Reports · doi:10.1177/00332941211025260 · Cited by 8
The Influence of Episodic Future Thinking and Graphic Warning Labels on Delay Discounting and Cigarette Demand
Naudé, G. P., Dolan, S. B., Strickland, J. C., Berry, M. S., Cox, D. J., & Johnson, M. W. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182312637 · Cited by 8
The (Mis)Understanding of the Symbol Associated with Recycling on Plastic Containers in the US: A Brief Report
Latkin, C. A., Dayton, L., Yi, G. T., & Balaban, A. (2022) · Sustainability · doi:10.3390/su14159636 · Cited by 8
Daily reciprocal relations between emotion dysregulation and non‐suicidal self‐injury among individuals with a history of sexual assault: The influence of posttraumatic stress symptoms
Raudales, A. M., Yang, M., Schatten, H. T., Armey, M. F., & Weiss, N. H. (2022) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12927 · Cited by 9
Anxiety Sensitivity Differentially Predicts Factors of Interpersonal–Psychological Suicide Risk: A Consideration of Specificity
Hudiburgh, S. E., Shaw, A. M., Hall, K. A. A., & Timpano, K. R. (2017) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12421 · Cited by 8
Media use and political learning: Comparing Trump supporters to celebrity candidate voters
Kanihan, S. F., & Rim, H. (2018) · Atlantic Journal of Communication · doi:10.1080/15456870.2018.1493881 · Cited by 8
Perceptions and Preferences of Laypersons in the Management of Positional Plagiocephaly
Vu, G. H., Magoon, K., Zimmerman, C. E., Kalmar, C. L., Humphries, L. S., Swanson, J. W., Bartlett, S. P. P., & Taylor, J. A. (2020) · Journal of Craniofacial Surgery · doi:10.1097/scs.0000000000006524 · Cited by 8
Crowdsourcing in observational and experimental research
Zallot, C., Paolacci, G., Chandler, J., & Sisso, I. (2021) · Routledge eBooks · doi:10.4324/9781003025245-12 · Cited by 8
Self-citation
The causal impact of objective numeracy on judgments: Improving numeracy via symbolic and non-symbolic approximate arithmetic training yields more consistent risk judgments
Chesney, D. L., Shoots‐Reinhard, B. L., & Peters, E. M. (2021) · Journal of Numerical Cognition · doi:10.5964/jnc.6925 · Cited by 8
Adverse Childhood Experiences as a Determinant of Public Service Motivation
Evans, C. A., & Evans, G. R. (2018) · Public Personnel Management · doi:10.1177/0091026018801043 · Cited by 11
Assessing Online Flow Across Cultures: A Two-Fold Measurement Invariance Study
Hu, E., Stavropoulos, V., Anderson, A., Clarke, M., Beard, C. L., Papapetrou, S., & Gomez, R. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00407 · Cited by 8
Psychological reactions to COVID-19: Survey data assessing perceived susceptibility, distress, mindfulness, and preventive health behaviors
O’Brien, W. H., Wang, S., Xu, H., Wang, S., Yang, Z., Yang, J. T., Liu, Q., Zhang, X., Tang, L., Varga, A. V., et al. (2020) · Data in Brief · doi:10.1016/j.dib.2020.106687 · Cited by 9
Dark user interface, dark behavior? The effect of ‘dark mode’ on honesty
Koning, L., & Junger, M. (2021) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2021.100107 · Cited by 7
Financial Anxiety Among Caregiving Parents of Adult Children with a Substance Use Disorder
Tambling, R. R., D’Aniello, C., & Russell, B. S. (2021) · Journal of Financial Therapy · doi:10.4148/1944-9771.1255 · Cited by 7
Preferences for Attributes of Initial COVID-19 Diagnosis in the United States and China During the Pandemic: Discrete Choice Experiment With Propensity Score Matching
Zhang, Y., Liu, T., He, Z., Chan, S. N., Akinwunmi, B. O., Huang, J. V., Wong, T.-H., Zhang, C. J. P., & Ming, W. (2022) · JMIR Public Health and Surveillance · doi:10.2196/37422 · Cited by 7
Temporal self and reflected appraisals in euthymic and depressed individuals
Sokol, Y., Levin, C., Rosensweig, C., Talasazan, N., & Serper, M. R. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.02.022 · Cited by 7
The Impact of Media Watching and Victim Gender on Victim and Offender Blameworthiness and Punishment
Schanz, K., & Jones, E. E. (2023) · Violence Against Women · doi:10.1177/10778012231170862 · Cited by 7
An Experimental Study to Examine Relationships Between IT Identity and Users’ Post-Adoption Behaviors for Different Types of Health Applications
Maddah, M., Esmaeilzadeh, P., & Mirzaei, T. (2023) · Information Systems Management · doi:10.1080/10580530.2023.2237187 · Cited by 9
Better social reversal learning is associated with a more social approach across time
Zabag, R., Kivity, Y., Gilboa‐Schechtman, E., & Levy‐Gigi, E. (2024) · Scientific Reports · doi:10.1038/s41598-024-58348-5 · Cited by 7
Within‐ and between‐person effects of causal attributions on relationship improvement following perceived incivility
Carson, J. E., Mackey, J. D., Alexander, K. N., McAllister, C. P., & Phillipich, M. A. (2024) · Journal of Occupational and Organizational Psychology · doi:10.1111/joop.12537 · Cited by 7
Evaluating the psychometric properties of the 24-item and 12-item real relationship inventory-client forms
Stefana, A., Fusar‐Poli, P., Vieta, E., & Youngstrom, E. A. (2025) · PLoS ONE · doi:10.1371/journal.pone.0311411 · Cited by 7
Participant Crosstalk: Issues When Using the Mechanical Turk
Edlund, J. E., Lange, K. M., Sevene, A. M., Umansky, J., Beck, C. D., & Bell, D. J. (2017) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.13.3.p174 · Cited by 7
Racial Differences in Anticipated Satisfaction With Life during Retirement
Tsotsoros, C. E., Hershey, D. A., Kiso, H., & Koposko, J. L. (2019) · Work Aging and Retirement · doi:10.1093/workar/waz002 · Cited by 7
The structure and function of coping in emerging adults
Peisch, V. D., & Burt, K. B. (2020) · Current Psychology · doi:10.1007/s12144-020-00990-z · Cited by 7
Factors Contributing to Parents' Psychological and Medical Help Seeking During the COVID-19 Global Pandemic
Tambling, R. R., Russell, B. S., Tomkunas, A. J., Horton, A. L., & Hutchison, M. (2021) · Family & Community Health · doi:10.1097/fch.0000000000000298 · Cited by 8
Nowe narzędzia i techniki zwiększające trafność badań internetowych
Saad, D. (2021) · Com press · doi:10.51480/compress.2021.4-1.248 · Cited by 7
Silence of crowdworkers—reasons and implications for work conditions and quality
Osterbrink, L., & Alpar, P. (2021) · International Studies of Management and Organization · doi:10.1080/00208825.2021.1927311 · Cited by 7
Social threat indirectly increases moral condemnation via thwarting fundamental social needs
Henderson, R. K., & Schnall, S. (2021) · Scientific Reports · doi:10.1038/s41598-021-00752-2 · Cited by 7
Video-Based Communication Assessment of Physician Error Disclosure Skills by Crowdsourced Laypeople and Patient Advocates Who Experienced Medical Harm: Reliability Assessment With Generalizability Theory
White, A. A., King, A., D’Addario, A. E., Brigham, K. B., Dintzis, S. M., Fay, E., Gallagher, T. H., & Mazor, K. M. (2022) · JMIR Medical Education · doi:10.2196/30988 · Cited by 7
The good, the bad, and the mixed: Experiences during COVID-19 among an online sample of adults
Mills, D. J., Petrovic, J., Mettler, J., Hamza, C. A., & Heath, N. L. (2022) · PLoS ONE · doi:10.1371/journal.pone.0269382 · Cited by 7
Hope and Thriving in the LGBTQ+ Community: Impact on the Interaction Between Sexual Identity and Outness
Goshorn, J. R., Sperandio, K. R., & Mason, W. N. (2022) · Journal of LGBTQ Issues in Counseling · doi:10.1080/26924951.2022.2054490 · Cited by 7
The pathways from online health information seeking to cyberchondria: A perspective from moderated mediation
Afrin, R., Prybutok, G. T., & Prybutok, V. R. (2022) · Psychiatry Research Communications · doi:10.1016/j.psycom.2022.100069 · Cited by 7
Pursuing, developing, and letting go of a passionate activity when facing adversity during a pandemic: Associations with well-being and ill-being
Paquette, V., Holding, A. C., Cimon‐Paquet, C., Giroux, A., Boucher, V. G., & Vallerand, R. J. (2023) · Personality and Individual Differences · doi:10.1016/j.paid.2023.112217 · Cited by 10
To pay or not to pay? handling crowdsourced participants who drop out from a research study
Benbunan‐Fich, R. (2023) · Ethics and Information Technology · doi:10.1007/s10676-023-09708-8 · Cited by 7
Introspective responses to cues and motivation to reduce cigarette smoking influence state and behavioral responses to cue exposure
Veilleux, J. C., & Skinner, K. D. (2016) · Addictive Behaviors · doi:10.1016/j.addbeh.2016.04.002 · Cited by 7
Racial Disparities in Cleanliness Attitudes Mediate Purchasing Attitudes Toward Cleaning Products: a Serial Mediation Model
Litman, L., Williams, M. T., Rosen, Z., Weinberger-Litman, S. L., & Robinson, J. (2017) · Journal of Racial and Ethnic Health Disparities · doi:10.1007/s40615-017-0429-y · Cited by 7
Disgust sensitivity and kosher food preferences among the non-Jewish population in the US
Hamerman, E. J., Schneider, A. B., & Rozensher, S. G. (2019) · Appetite · doi:10.1016/j.appet.2019.104413 · Cited by 7
COVID-19—Related Assault on Asians: Economic Hardship in the United States and India Predicts Diminished Support for Victim Compensation and Assailant Punishment
Johnson, J. D., Sattler, D. N., & Otton, K. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18105320 · Cited by 7
Family experiences of ambiguous loss during COVID ‐19
Horton, A. L., Russell, B. S., Tambling, R. R., Elias, H., & Mas, M. (2023) · Family Relations · doi:10.1111/fare.12918 · Cited by 7
How should the severity of depression be rated on self-report depression scales?
Zimmerman, M. A., & Kerr, S. (2019) · Psychiatry Research · doi:10.1016/j.psychres.2019.112512 · Cited by 7
Factors Affecting Patient Decision-Making Regarding Midshaft Clavicle Fracture Treatment
Pérez, G. M., Tran, M. M., McDonald, C. L., O’Donnell, R., & Cruz, A. I. (2020) · Cureus · doi:10.7759/cureus.10505 · Cited by 7
Coping with COVID-19: Survey data assessing psychological distress to COVID-19 and vaccine hesitancy with measures of theory of planned behavior, mindfulness, compassion, cultural orientation, and pandemic fatigue
Mueller, E. A., Suvanbenjakule, P., Lim, C. X., O’Brien, W. H., Chavanovanich, J., Jarukasemthawee, S., Pisitsungkagarn, K., & Suavansri, P. (2022) · Data in Brief · doi:10.1016/j.dib.2022.108390 · Cited by 7
Factors in predicting consumer affinity for non-alcoholic beer in the US
Hamerman, E. J., & Aggarwal, A. (2025) · Appetite · doi:10.1016/j.appet.2025.108192 · Cited by 6
Exploring the role and challenges of AI tools in physical education: qualitative insights from in-service, trainee, and future teachers
Bofill-Herrero, A., García-Taibo, O., Férriz-Valero, A., & Baena‐Morales, S. (2025) · Sport Education and Society · doi:10.1080/13573322.2025.2547869 · Cited by 6
Estimating the smallest worthwhile difference of recommended psychotherapies for depression: observational study
Sahker, E., Luo, Y., Omae, K., Tajika, A., Ferreira, M. L., Chevance, A., Leucht, S., Markham, S., Ede, R., Cipriani, A., et al. (2025) · The British Journal of Psychiatry · doi:10.1192/bjp.2025.10453 · Cited by 6
The Interactive Effect of Anxiety Sensitivity and Metacognitive Beliefs on Anxiety
Gorday, J. Y., & Bardeen, J. R. (2022) · Journal of Cognitive Psychotherapy · doi:10.1891/jcpsy-d-20-00054 · Cited by 6
Can Faking Be Measured With Dedicated Validity Scales? Within-Subject Trifactor Mixture Modeling Applied to BIDR Responses
Guenole, N., Brown, A., & Lim, V. (2022) · Assessment · doi:10.1177/10731911221098434 · Cited by 6
The Individual Time Span Scales (ITSS): Introducing a self-report questionnaire to measure individual differences in the perception of time as duration
Nowack, K. (2023) · Personality and Individual Differences · doi:10.1016/j.paid.2023.112116 · Cited by 6
The shift in sports viewing: scale development and validation for mediated sports consumption motivation
Kim, M. J., Mao, L. L., Seidler, T. L., & Barnes, J. C. (2024) · European Sport Management Quarterly · doi:10.1080/16184742.2024.2329157 · Cited by 7
Depression and Anxiety in Older Adults: A Comparison Between Online Convenience and Conventional Representative Sampling
Georgi, H. S., Dragomirecká, E., Tichá, Z., & Mana, J. (2025) · Depression and Anxiety · doi:10.1155/da/2522358 · Cited by 7
The role of disgust sensitivity in volunteer recruitment and retention
Hamerman, E. J., & Schneider, A. B. (2017) · International Journal of Nonprofit and Voluntary Sector Marketing · doi:10.1002/nvsm.1597 · Cited by 8
Creative Action at a Distance: A Conceptual Framework for Embodied Performance With Robotic Actors
Wicke, P., & Veale, T. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.662182 · Cited by 6
Establishing initial validity and factor structure for the muscularity-oriented eating test in gay men
Donahue, J. M., Scharmer, C., Fogarty, S., & Walker, D. C. (2022) · Eating Behaviors · doi:10.1016/j.eatbeh.2022.101631 · Cited by 6
The original sin of crowd work for human subjects research
Xia, H. (2022) · Journal of Information Communication and Ethics in Society · doi:10.1108/jices-12-2021-0126 · Cited by 6
Do You Mind? Examining the Impact of Psychoeducation Specificity on Perceptions of Mindfulness-Based Programs
Rosario, N. D., & Beshai, S. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19159621 · Cited by 6
Does Meaning-in-Life or Self-Compassion Influence LGBTQ+ Identity or Outness?
Goshorn, J. R., Mason, W. N., & Sperandio, K. R. (2022) · Journal of LGBTQ Issues in Counseling · doi:10.1080/26924951.2022.2093309 · Cited by 6
COVID-19-related threat, existential isolation, and well-being
Vail, K. E., Galgali, M. S., Reed, D. E., Helm, P. J., Edwards, M., Jimenez, T., Arndt, J., Lehinger, E. A., Sedivy, L. P., McGeary, D. D., et al. (2023) · Clinical Psychologist · doi:10.1080/13284207.2022.2155510 · Cited by 6
Integrating Self-Report and Psychophysiological Measures in Waterpipe Tobacco Message Testing: A Novel Application of Multi-Attribute Decision Modeling
Stevens, E. M., Villanti, A. C., Leshner, G., Wagener, T. L., Keller-Hamilton, B. L., & Mays, D. M. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182211814 · Cited by 6
Political and Social Drivers of COVID-19 Prevention and Climate Change Behaviors and Attitudes
Latkin, C. A., Hendrickson, Z. M., Dayton, L., & Bonneau, H. (2023) · Climate · doi:10.3390/cli11030053 · Cited by 6
Sleep is the best medicine: assessing sleep, disordered eating, and weight-related functioning
Barnes, R. D., Palmer, B. E., Hanson, S., & Lawson, J. L. (2023) · Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity · doi:10.1007/s40519-023-01610-y · Cited by 6
Development of the Japanese version of the Visual Discomfort Scale
Imaizumi, S., Koyama, S., & Tanno, Y. (2018) · PLoS ONE · doi:10.1371/journal.pone.0191094 · Cited by 6
Biological causal beliefs and depression stigma: the moderating effects of first- and second-hand experience with depression
Mann, S., & Contrada, R. J. (2020) · Journal of Mental Health · doi:10.1080/09638237.2020.1755018 · Cited by 6
Parents' Well-Being and Emotion Regulation During Infancy: The Mediating Effects of Coping
Russell, B. S., Adamsons, K. L., Hutchison, M., & Francis, J. (2021) · The Family Journal · doi:10.1177/10664807211022230 · Cited by 10
Intertemporal Differences Among MTurk WorkersPreprint
Casey, L. S., & Chandler, J. (2016) · doi:10.31234/osf.io/8352x · Cited by 6
Self-citation
Misinformation about COVID-19: Evidence for Differential Latent Profiles and a Strong Association with Trust in SciencePreprint
Agley, J. D., & Xiao, Y. (2020) · doi:10.21203/rs.3.rs-35919/v2 · Cited by 6
Incoming Editorial
Samuel, D. B. (2019) · Assessment · doi:10.1177/1073191118822760 · Cited by 6
Moral Polarization and Out-Party Hostility in the US Political ContextPreprint
Tappin, B. M., & McKay, R. (2017) · doi:10.31234/osf.io/4fxb3 · Cited by 6
Factors Shaping Public Perceptions of a Range of Robotic Technologies in Surgery: Cross-Sectional Web-Based Survey
Shen, S. A., Xie, D. X., Ding, A. S., Zhang, L., & Creighton, F. X. (2025) · JMIR Human Factors · doi:10.2196/64224 · Cited by 5
Cannabis Use Disorder Treatment Preferences: A Pilot Survey in Current Users of Cannabis
Lile, J. A., Turner, B. W., Cox, D. H., Bonn‐Miller, M. O., Katz, N. R., Shellenberg, T., Stoops, W. W., & Strickland, J. C. (2022) · Journal of Addiction Medicine · doi:10.1097/adm.0000000000001059 · Cited by 5
Reinforcement-Learning-Informed Queries Guide Behavioral Change
Brown, V. M., Lee, J., Wang, J., Casas, B. R., & Chiu, P. H. (2024) · Clinical Psychological Science · doi:10.1177/21677026231213368 · Cited by 5

Authors: Chandler, J., & Shapiro, D.
Affiliations: Mathematica Policy Research (US)
University of Michigan (US)
ORCID: Chandler, Jesse
Venue: Annual Review of Clinical Psychology
Publisher: Annual Reviews
Published: 2016-03-28
Volume: 12
Issue: 1
Pages: 53-81
ISSN: 1548-5943 (ISSN-L)
1548-5951
DOI: 10.1146/annurev-clinpsy-021815-093623
OpenAlex: W2126493218
Cited by: 1,072
References: 177
Citations by year: 2026: 27 · 2025: 70 · 2024: 105 · 2023: 103 · 2022: 150 · 2021: 168 · 2020: 122 · 2019: 115 · 2018: 111 · 2017: 77 · 2016: 24
FWCI: 17.58
Type: journal article
Language: en
Open access: closed
Locations: Annual Review of Clinical Psychology (publishedVersion)
PubMed (publishedVersion)
RePEc: Research Papers in Economics (submittedVersion)
Topics: Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
Keywords: crowdsourcing, Amazon Mechanical Turk, convenience samples, non-probability sampling, sample composition, data quality, participant recruitment, digital labour markets
MeSH terms: Humans, Biomedical Research, Crowdsourcing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Conducting Clinical Research Using Crowdsourced Convenience Samples

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks