Cited by 1,072Mobile Crowdsensing and CrowdsourcingSurvey Methodology and NonresponseEthics in Clinical Research

The DOI matches its Crossref record; 174 of the 177 references it cites were found in a registry or an index.

Conducting Clinical Research Using Crowdsourced Convenience Samples

Chandler, Jesse, Shapiro, Danielle·2016·Annual Review of Clinical Psychology

DOI 10.1146/annurev-clinpsy-021815-093623

Open this paper in CoChat — free account

Abstract

Crowdsourcing has had a dramatic impact on the speed and scale at which scientific research can be conducted. Clinical scientists have particularly benefited from readily available research study participants and streamlined recruiting and payment systems afforded by Amazon Mechanical Turk (MTurk), a popular labor market for crowdsourcing workers. MTurk has been used in this capacity for more than five years. The popularity and novelty of the platform have spurred numerous methodological investigations, making it the most studied nonprobability sample available to researchers. This article summarizes what is known about MTurk sample composition and data quality with an emphasis on findings relevant to clinical psychological research. It then addresses methodological issues with using MTurk--many of which are common to other nonprobability samples but unfamiliar to clinical science researchers--and suggests concrete steps to avoid these issues or minimize their impact.

Integrity signals

Registry

Crossref · 10.1146/annurev-clinpsy-021815-093623

Cites

174 of 174 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Self-citation
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Self-citation
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Ed, H., & Suh, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,941

Show all 177 references

Research works printed but not linked: 12

Barnhoorn JS, 2014, Behav. Res. Methods, 530, 1
Beramendi P, Duch RM, Matsuo A. 2014. When lab subjects meet real people: comparing different modes of experiments. Presented at Asian Polit. Methodol. Conf., Taipai
Casey L, Chandler J, Levine AS, Proctor A, Strolovitch D. 2015. Demographic characteristics of a large sample of US workers. Mathematica Policy Res. Rep. In press
Downes-Le Guin T, 2006, ESOMAR World Res. Conf. Panel Res.
Higgins C, 2010, Proc. NAACL HLT 2010 Workshop on Creat. Speech and Language Data with Amazon's Mechanical Turk, Los Angeles, 89
10.1007/s10683-011-9273-9 · DOI
10.1145/2676431.2676437 · DOI
Lang ASID, 2011, Code4Lib J., 15, 1
Lebreton P, 2015, Proc. SPIE Conf. Hum. Vis. Electron. Imaging, 20th, San Francisco
10.1026//1618-3169.49.4.243 · DOI
Shao W, 2015, Digit. Cult. Educ., 7
Suri S, 2011, AAAI Workshops, 25th, San Francisco, 61

Sorted by citations

Items 101–200 of 1,001

The Protective Role of Parent Resilience on Mental Health and the Parent–Child Relationship During COVID-19
Russell, B. S., Tomkunas, A. J., Hutchison, M., Tambling, R. R., & Horton, A. L. (2021) · Child Psychiatry & Human Development · doi:10.1007/s10578-021-01243-1 · Cited by 80
Associations of parents' self, child, and other “fat talk” with child eating behaviors and weight
Lydecker, J. A., Riley, K. E., & Grilo, C. M. (2018) · International Journal of Eating Disorders · doi:10.1002/eat.22858 · Cited by 67
Crowdsourced data collection for public health: A comparison with nationally representative, population tobacco use data
Kraemer, J. D., Strasser, A. A., Lindblom, E. N., NIAURA, R. S., & Mays, D. M. (2017) · Preventive Medicine · doi:10.1016/j.ypmed.2017.07.006 · Cited by 66
Kratom use as more than a “self-treatment”
Smith, K. E., Dunn, K. E., Rogers, J. M., Grundmann, O., McCurdy, C. R., Garcia‐Romeu, A., Schriefer, D., Swogger, M. T., & Epstein, D. H. (2022) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2022.2083967 · Cited by 66
Self-construal and gender interact to cause social evaluative concerns
Russell, L. H., Gould, K., & Fergus, T. A. (2016) · Personality and Individual Differences · doi:10.1016/j.paid.2016.12.041 · Cited by 65
Examining emotional functioning in misophonia: The role of affective instability and difficulties with emotion regulation
Guetta, R. E., Cassiello-Robbins, C., Trumbull, J., Anand, D., & Rosenthal, M. Z. (2022) · PLOS ONE · doi:10.1371/journal.pone.0263230 · Cited by 64
Conducting organizational survey and experimental research online: From convenient to ambitious in study designs, recruiting, and data quality
Quaquebeke, N. V., Salem, M., van Dijke, M., & Wenzel, R. (2022) · Organizational Psychology Review · doi:10.1177/20413866221097571 · Cited by 75
Phytophilia and Service Atmospherics: The Effect of Indoor Plants on Consumers
Tifferet, S., & Vilnai‐Yavetz, I. (2016) · Environment and Behavior · doi:10.1177/0013916516669390 · Cited by 65
Emotion Regulation as a Mediator of Self-Compassion and Depressive Symptoms in Recurrent Depression
Bakker, A. M., Cox, D. W., Hubley, A. M., & Owens, R. L. (2019) · Mindfulness · doi:10.1007/s12671-018-1072-3 · Cited by 61
Estimates of Non-Heterosexual Prevalence: The Roles of Anonymity and Privacy in Survey Methodology
Robertson, R. E., Tran, F. W., Lewark, L. N., & Epstein, R. (2017) · Archives of Sexual Behavior · doi:10.1007/s10508-017-1044-z · Cited by 61
Improving the Statistical Power and Reliability of Research Using Amazon Mechanical Turk
Bentley, J. W. (2021) · Accounting Horizons · doi:10.2308/horizons-18-052 · Cited by 61
Exposing omitted moderators: Explaining why effect sizes differ in the social sciences
Krefeld-Schwalb, A., Sugerman, E. R., & Johnson, E. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2306281121 · Cited by 60
Safety behaviors, experiential avoidance, and anxiety: A path analysis approach
Kirk, A., Meyer, J. M., Whisman, M. A., Deacon, B. J., & Arch, J. J. (2019) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2019.03.002 · Cited by 59
Evaluating Tutorial-Based Instructions for Controllers in Virtual Reality Games
Kao, D., Magana, A. J., & Mousas, C. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3474661 · Cited by 60
Quantifying the Impact of Public Perceptions on Vaccine Acceptance Using Behavioral Economics
Hursh, S. R., Strickland, J. C., Schwartz, L. P., & Reed, D. D. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.608852 · Cited by 58
Behavioral economic methods to inform infectious disease response: Prevention, testing, and vaccination in the COVID-19 pandemic
Strickland, J. C., Reed, D. D., Hursh, S. R., Schwartz, L. P., Foster, R. N. S., Gelino, B. W., LeComte, R. S., Oda, F. S., Salzer, A. R., Schneider, T. D., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0258828 · Cited by 63
Self-identification as a pornography addict: examining the roles of pornography use, religiousness, and moral incongruence
Grubbs, J. B., Weinandy, J. T. G., & Engelman, J. (2018) · Sexual Addiction & Compulsivity The Journal of Treatment and Prevention · doi:10.1080/10720162.2019.1565848 · Cited by 58
Using Mechanical Turk to recruit participants for internet intervention research: experience from recruitment for four trials targeting hazardous alcohol consumption
Cunningham, J. A., Godinho, A., & Kushnir, V. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0440-3 · Cited by 57
Do imposters threaten data quality? An examination of worker misrepresentation and downstream consequences in Amazon's Mechanical Turk workforce
Kan, I. P., & Drummey, A. B. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.02.005 · Cited by 62
Mental distress among U.S. adults during the COVID-19 pandemic
Twenge, J., & Joiner, T. (2020) · doi:10.31234/osf.io/wc8ud · Cited by 330
Predicting health behavior in response to the coronavirus disease (COVID-19): Worldwide survey results from early March 2020
Anaki, D., & Sergay, J. A. (2021) · PLoS ONE · doi:10.1371/journal.pone.0244534 · Cited by 56
A theoretical model of cyberchondria development: Antecedents and intermediate processes
Zheng, H., Kim, H. K., Sin, S. J., & Theng, Y. (2021) · Telematics and Informatics · doi:10.1016/j.tele.2021.101659 · Cited by 56
Machine Learning Approaches for Depression Detection on Social Media: A Systematic Review of Biases and Methodological Challenges.
Cao, Y., Dai, J., Wang, Z., Zhang, Y., Shen, X., Liu, Y., & Tian, Y. (2025) · Journal of Behavioral Data Science · doi:10.35566/jbds/caoyc · Cited by 62
The Difficulties in Emotion Regulation Scale–Parent Report: A Psychometric Investigation Examining Adolescents With and Without ADHD
Bunford, N., Dawson, A. E., Evans, S. W., Ray, A., Langberg, J. M., Owens, J. S., DuPaul, G. J., & Allan, D. M. (2018) · Assessment · doi:10.1177/1073191118792307 · Cited by 59
Crowdsourcing for Online Samples in Counseling Research
Mullen, P. R., Fox, J., Goshorn, J. R., & Warraich, L. K. (2021) · Journal of Counseling & Development · doi:10.1002/jcad.12369 · Cited by 56
Trust in experts, not trust in national leadership, leads to greater uptake of recommended actions during the COVID‐19 pandemic
Ahluwalia, S. C., Edelen, M. O., Qureshi, N., & Etchegaray, J. M. (2021) · Risk Hazards & Crisis in Public Policy · doi:10.1002/rhc3.12219 · Cited by 61
The Influence of Gamification and Information Technology Identity on Postadoption Behaviors of Health and Fitness App Users: Empirical Study in the United States
Esmaeilzadeh, P. (2021) · JMIR Serious Games · doi:10.2196/28282 · Cited by 55
Searching for a Signal: Self-Reported Kratom Dose-Effect Relationships Among a Sample of US Adults With Regular Kratom Use Histories
Smith, K. E., Rogers, J. M., Dunn, K. E., Grundmann, O., McCurdy, C. R., Schriefer, D., & Epstein, D. H. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.765917 · Cited by 54
Institutional courage buffers against institutional betrayal, protects employee health, and fosters organizational commitment following workplace sexual harassment
Smidt, A. M., Adams-Clark, A. A., & Freyd, J. J. (2023) · PLoS ONE · doi:10.1371/journal.pone.0278830 · Cited by 61
The potential of digital nudging to bridge the gap between environmental attitude and behavior in the usage of smart home applications
Stieglitz, S., Mirbabaie, M., Deubel, A., Braun, L.-M., & Kißmer, T. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102665 · Cited by 53
Predicting pornography use over time: Does self-reported “addiction” matter?
Grubbs, J. B., Wilt, J. A., Exline, J. J., & Pargäment, K. I. (2018) · Addictive Behaviors · doi:10.1016/j.addbeh.2018.02.028 · Cited by 53
FFMPD scales: Comparisons with the FFM, PID-5, and CAT-PD-SF.
Crego, C., Oltmanns, J. R., & Widiger, T. A. (2018) · Psychological Assessment · doi:10.1037/pas0000495 · Cited by 52
Conflict and cooperation in paranoia: a large-scale behavioural experiment
Raihani, N., & Bell, V. (2017) · Psychological Medicine · doi:10.1017/s0033291717003075 · Cited by 52
National norms for the expanded version of the inventory of depression and anxiety symptoms (IDAS-II)
Nelson, G. H., O'Hara, M. W., & Watson, D. (2018) · Journal of Clinical Psychology · doi:10.1002/jclp.22560 · Cited by 51
Online Crowdsourcing to Explore Public Perceptions of Robotic-Assisted Orthopedic Surgery
Pagani, N. R., Moverman, M. A., Puzzitiello, R. N., Menendez, M. E., Barnes, C. L., & Kavolus, J. J. (2021) · The Journal of Arthroplasty · doi:10.1016/j.arth.2021.02.027 · Cited by 51
Stigma and recognition of different types of symptoms in OCD
McCarty, R. J., Guzick, A. G., Swan, L. K., & McNamara, J. P. H. (2016) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2016.12.006 · Cited by 51
Evaluating non-medical prescription opioid demand using commodity purchase tasks: test-retest reliability and incremental validity
Strickland, J. C., Lile, J. A., & Stoops, W. W. (2019) · Psychopharmacology · doi:10.1007/s00213-019-05234-y · Cited by 51
Social, psychological, and substance use characteristics of U.S. adults who use kratom: Initial findings from an online, crowdsourced study.
Smith, K. E., Dunn, K. E., Grundmann, O., Garcia‐Romeu, A., Rogers, J. M., Swogger, M. T., & Epstein, D. H. (2021) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000518 · Cited by 51
How Perceptions of Trust, Risk, Tap Water Quality, and Salience Characterize Drinking Water Choices
Grupper, M. A., Schreiber, M. E., & Sorice, M. G. (2021) · Hydrology · doi:10.3390/hydrology8010049 · Cited by 50
Perceived Partner Responsiveness Moderates the Associations Between Sexual Talk and Sexual and Relationship Well-Being in Individuals in Long-Term Relationships
Merwin, K. E., & Rosen, N. O. (2019) · The Journal of Sex Research · doi:10.1080/00224499.2019.1610151 · Cited by 50
Using Mechanical Turk to Study Parents and Children: An Examination of Data Quality and Representativeness
Jensen‐Doss, A., Patel, Z. S., Casline, E. P., Ringle, V. A. M., & Timpano, K. R. (2021) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2020.1815205 · Cited by 50
Counting polyamorists who count: Prevalence and definitions of an under-researched form of consensual nonmonogamy
Rubel, A. N., & Burleigh, T. J. (2018) · Sexualities · doi:10.1177/1363460718779781 · Cited by 49
Traditional and Modern Convenience Samples: An Investigation of College Student, Mechanical Turk, and Mechanical Turk College Student Samples
Weigold, A., & Weigold, I. K. (2021) · Social Science Computer Review · doi:10.1177/08944393211006847 · Cited by 49
Developing a novel assessment of interpretation flexibility: Reliability, validity and clinical implications
Deng, W., Everaert, J., Creighton, M., Bronstein, M. V., Cannon, T. D., & Joormann, J. (2022) · Personality and Individual Differences · doi:10.1016/j.paid.2022.111548 · Cited by 50
Oh the Anxiety! The Anxiety of Supervisor Bottom-Line Mentality and Mitigating Effects of Ethical Leadership
Mitchell, M. S., Hetrick, A. L., Mawritz, M. B., Edwards, B., & Greenbaum, R. L. (2023) · Journal of Management · doi:10.1177/01492063231196553 · Cited by 52
Emotions and Cognitions Associated with the Stigma of Non-Offending Pedophilia: A Vignette Experiment
Jahnke, S. (2017) · Archives of Sexual Behavior · doi:10.1007/s10508-017-1073-7 · Cited by 49
Careless responding in crowdsourced alcohol research: A systematic review and meta-analysis of practices and prevalence.
Jones, A., Earnest, J., Adam, M., Clarke, R., Yates, J., & Pennington, C. R. (2022) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000546 · Cited by 49
The Use of Online Panel Data in Management Research: A Review and Recommendations
Porter, C. O. L. H., Outlaw, R., Gale, J. P., & Cho, T. S. (2018) · Sage Journals Data · doi:10.25384/sage.c.4310936 · Cited by 49
No pain no gain? Social demographic correlates and identity consequences of interpreting experienced difficulty as importance
Aelenei, C., Lewis, N. A., & Oyserman, D. (2016) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2016.08.004 · Cited by 48
Fatalism and Depressive Symptoms: Active and Passive Forms of Fatalism Differentially Predict Depression
Shahid, F., Beshai, S., & Rosario, N. D. (2020) · Journal of Religion and Health · doi:10.1007/s10943-020-01024-5 · Cited by 48
Negative affect interference and fear of happiness are independently associated with depressive symptoms
Jordan, D. G., Collins, A. C., Dunaway, M. G., Kilgore, J., & Winer, E. S. (2020) · Journal of Clinical Psychology · doi:10.1002/jclp.23066 · Cited by 48
Development and Validation of the Four Facet Mindful Eating Scale (FFaMES)
Carrière, K., Shireen, S. H., Siemers, N., Preißner, C. E., Starr, J., Falk, C., & Knäuper, B. (2022) · Appetite · doi:10.1016/j.appet.2021.105689 · Cited by 47
Examining components of emotion regulation in relation to sleep problems and suicide risk
Ward-Ciesielski, E. F., Winer, E. S., Drapeau, C. W., & Nadorff, M. R. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.07.065 · Cited by 47
Internalized versus externalized continuing bonds: Relations to grief, trauma, attachment, openness to experience, and posttraumatic growth
Black, J., Belicki, K., Emberley-Ralph, J., & McCann, A. (2020) · Death Studies · doi:10.1080/07481187.2020.1737274 · Cited by 53
Feasibility, acceptability, and validity of crowdsourcing for collecting longitudinal alcohol use data
Strickland, J. C., & Stoops, W. W. (2018) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.445 · Cited by 47
Human roars communicate upper-body strength more effectively than do screams or aggressive and distressed speech
Raine, J., Pisanski, K., Bond, R., Simner, J., & Reby, D. (2019) · PLoS ONE · doi:10.1371/journal.pone.0213034 · Cited by 53
What predicts climate change activism?: An examination of how depressive symptoms, climate change distress, and social norms are associated with climate change activism
Latkin, C. A., Dayton, L., Scherkoske, M., Countess, K., & Thrul, J. (2022) · The Journal of Climate Change and Health · doi:10.1016/j.joclim.2022.100146 · Cited by 46
A framework of an integrated sustainability and value engineering concepts for construction projects
Gunarathne, A. S., Zainudeen, N., Perera, C. S. R., & Perera, B. A. K. S. (2020) · International Journal of Construction Management · doi:10.1080/15623599.2020.1768624 · Cited by 46
Me, myself and money: having a financially focused self-concept and its consequences for disordered gambling
Tabri, N., Wohl, M. J. A., Eddy, K. T., & Thomas, J. J. (2016) · International Gambling Studies · doi:10.1080/14459795.2016.1252414 · Cited by 46
Online Methods in Adolescent Self-Injury Research: Challenges and Recommendations
Smith, D. M. Y., Lipson, S. M., Wang, S. B., & Fox, K. R. (2021) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2021.1875325 · Cited by 47
Using Mechanical Turk for research on cancer survivors
Arch, J. J., & Carr, A. L. (2016) · Psycho-Oncology · doi:10.1002/pon.4173 · Cited by 45
Initial evidence that non-clinical autistic traits are associated with lower income
Skylark, W. J., & Baron‐Cohen, S. (2017) · Molecular Autism · doi:10.1186/s13229-017-0179-z · Cited by 45
Disgust as an emotional driver of vaccine attitudes and uptake? A mediation analysis
Luz, P. M., Brown, H. E., & Struchiner, C. J. (2019) · Epidemiology and Infection · doi:10.1017/s0950268819000517 · Cited by 45
Dating app use and unhealthy weight control behaviors among a sample of U.S. adults: a cross-sectional study
Tran, A. H., Suharlim, C., Mattie, H., Davison, K. K., Agénor, M., & Austin, S. B. (2019) · Journal of Eating Disorders · doi:10.1186/s40337-019-0244-4 · Cited by 45
Women’s Disempowerment and Preferences for Skin Lightening Products That Reinforce Colorism: Experimental Evidence From India
Adbi, A., Chatterjee, C., Cortland, C. I., Kinias, Z., & Singh, J. (2021) · Psychology of Women Quarterly · doi:10.1177/0361684321993796 · Cited by 44
Socially stigmatized company’s CSR efforts during the COVID-19 pandemic: The effects of CSR fit and perceived motives
Lee, Y.-J., & Cho, M. (2022) · Public Relations Review · doi:10.1016/j.pubrev.2022.102180 · Cited by 44
The effects of juiciness in an action RPG
Kao, D. (2020) · Entertainment Computing · doi:10.1016/j.entcom.2020.100359 · Cited by 48
Toward human-centered AI management: Methodological challenges and future directions
Dong, M., Bonnefon, J., & Rahwan, I. (2024) · Technovation · doi:10.1016/j.technovation.2024.102953 · Cited by 47
Coverage of theDSM-IV-TR/DSM-5Section II Personality Disorders With theDSM-5Dimensional Trait Model
Rojas, S. L., & Widiger, T. A. (2016) · Journal of Personality Disorders · doi:10.1521/pedi_2016_30_262 · Cited by 42
Living, fast and slow: Is life history orientation associated with risk-related personality traits, risk attitudes, criminal outcomes, and gambling?
Mishra, S., Templeton, A. J., & Meadows, T. J. S. (2017) · Personality and Individual Differences · doi:10.1016/j.paid.2017.06.009 · Cited by 42
The Relationship Among Online Sexually Explicit Material Exposure to, Desire for, and Participation in Rough Sex
Vogels, E. A., & O’Sullivan, L. F. (2018) · Archives of Sexual Behavior · doi:10.1007/s10508-018-1290-8 · Cited by 42
Inflexible Interpretations of Ambiguous Social Situations: A Novel Predictor of Suicidal Ideation and the Beliefs That Inspire It
Everaert, J., Bronstein, M. V., Cannon, T. D., Klonsky, E. D., & Joormann, J. (2021) · Clinical Psychological Science · doi:10.1177/2167702621993867 · Cited by 42
A longitudinal examination of the role of attentional control in the relationship between posttraumatic stress and threat-related attentional bias: An eye-tracking study
Bardeen, J. R., & Daniel, T. A. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2017.09.003 · Cited by 42
Retention of participants recruited to a multi-year longitudinal study via Prolific
Kothe, E. J., & Ling, M. (2019) · doi:10.31234/osf.io/5yv2u · Cited by 42
Using AI chatbots (e.g., CHATGPT) in seeking health-related information online: The case of a common ailment
Esmaeilzadeh, P., Maddah, M., & Mirzaei, T. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100127 · Cited by 47
Can Amazon's Mechanical Turk be used to recruit participants for internet intervention trials? A pilot study involving a randomized controlled trial of a brief online intervention for hazardous alcohol use
Cunningham, J. A., Godinho, A., & Kushnir, V. (2017) · Internet Interventions · doi:10.1016/j.invent.2017.08.005 · Cited by 41
Proactive Personality and Decent Work Among Racially and Ethnically Diverse Working Adults
Douglass, R. P., Autin, K. L., Büyükgöze‐Kavas, A., & Gensmer, N. P. (2019) · Journal of Career Assessment · doi:10.1177/1069072719894571 · Cited by 41
To Provide or Protect? Masculinity, Economic Precarity, and Protective Gun Ownership in the United States
Warner, T. D., Tober, T. L., Bridges, T. S., & Warner, D. F. (2021) · Sociological Perspectives · doi:10.1177/0731121421998406 · Cited by 44
Sexism, Moral Disengagement, and Dark Triad Traits on Perpetrators of Sexual Violence Against Women and Community Men
Navas, M. P., Maneiro, L., Cutrín, O., Fraguela, J. A. G., & Sobral, J. (2021) · Sexual Abuse · doi:10.1177/10790632211051689 · Cited by 40
Predictors of Mobile Phone and Social Networking Site Dependency in Adulthood
Burnell, K., & Kuther, T. L. (2016) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2016.0209 · Cited by 44
Intolerance of uncertainty fuels depressive symptoms through rumination: Cross-sectional and longitudinal studies
Huang, V., Yu, M., Carleton, R. N., & Beshai, S. (2019) · PLoS ONE · doi:10.1371/journal.pone.0224865 · Cited by 50
An empirical assessment of common or usual names to label cell‐based seafood products
Hallman, W. K., & Hallman, W. K. (2020) · Journal of Food Science · doi:10.1111/1750-3841.15351 · Cited by 40
Me, Myself, and Money II: Relative Deprivation Predicts Disordered Gambling Severity via Delay Discounting, Especially Among Gamblers Who Have a Financially Focused Self-Concept
Tabri, N., Shead, N. W., & Wohl, M. J. A. (2017) · Journal of Gambling Studies · doi:10.1007/s10899-017-9673-7 · Cited by 40
Specificity of Affective Responses in Misophonia Depends on Trigger Identification
Savard, M.-A., Sares, A. G., Coffey, E. B. J., & Deroche, M. L. D. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.879583 · Cited by 39
Workplace availability, risk group and perceived barriers predictive of 2016–17 influenza vaccine uptake in the United States: A cross-sectional study
Luz, P. M., Johnson, R. E., & Brown, H. E. (2017) · Vaccine · doi:10.1016/j.vaccine.2017.08.078 · Cited by 39
The role of disgust in body image disturbance: Incremental predictive power of self-disgust
Stasik-O’Brien, S. M., & Schmidt, J. P. (2018) · Body Image · doi:10.1016/j.bodyim.2018.08.011 · Cited by 39
Relation of positive memory recall count and accessibility with post-trauma mental health
Contractor, A. A., Banducci, A. N., Dolan, M., Keegan, F. S., & Weiss, N. H. (2019) · Memory · doi:10.1080/09658211.2019.1628994 · Cited by 43
Behavioral and Attitudinal Correlates of Trusted Sources of COVID-19 Vaccine Information in the US
Latkin, C. A., Dayton, L., Miller, J., Yi, G. T., Jaleel, A., Nwosu, C. C., Yang, P., & Falade-Nwulia, O. O. (2021) · Behavioral Sciences · doi:10.3390/bs11040056 · Cited by 39
Outpatient Tinnitus Clinic, Self-Help Web Platform, or Mobile Application to Recruit Tinnitus Study Samples?
Probst, T., Pryss, R. C., Langguth, B., Spiliopoulou, M., Landgrebe, M., Vesala, M., Harrison, S., Schobel, J., Reichert, M., Stach, M., et al. (2017) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2017.00113 · Cited by 39
The Metacognitions Questionnaire–30: An Examination of a Bifactor Model and Measurement Invariance Among Men and Women in a Community Sample
Fergus, T. A., & Bardeen, J. R. (2017) · Assessment · doi:10.1177/1073191116685807 · Cited by 39
Frustration and ennui among Amazon MTurk workers
Fowler, C., Jiao, J., & Pitts, M. J. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01955-9 · Cited by 41
Designing Graphs that Promote Both Risk Understanding and Behavior Change
Okan, Y., Stone, E. R., & de Bruin, W. B. (2017) · Risk Analysis · doi:10.1111/risa.12895 · Cited by 38
Causal Inference Methods: Lessons from Applied Microeconomics
Dague, L., & Lahey, J. (2018) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/muy067 · Cited by 40
Getting shredded: Development and validation of a measure of muscularity-oriented disordered eating.
Cooper, M., Griffiths, K. M., & Burns, R. A. (2020) · Psychological Assessment · doi:10.1037/pas0000804 · Cited by 38
Longitudinal evaluation of anhedonia as a mediator of fear of positive evaluation and other depressive symptoms
Jordan, D. G., Winer, E. S., Salem, T., & Kilgore, J. (2017) · Cognition & Emotion · doi:10.1080/02699931.2017.1289895 · Cited by 38
The smartphone in the memory clinic: A study of patient and care partner’s utilisation habits
Benge, J. F., Dinh, K. L., Logue, E. M., Phenis, R. A., Dasse, M. N., & Scullin, M. K. (2018) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2018.1459307 · Cited by 37
Creativity on Paid Crowdsourcing Platforms
Oppenlaender, J., Milland, K., Visuri, A., Ipeirotis, P. G., & Hosio, S. J. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376677 · Cited by 42
Autobiographical memory coherence in emotional disorders: The role of rumination, cognitive avoidance, executive functioning, and meaning making
Vanderveren, E., Bijttebier, P., & Hermans, D. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231862 · Cited by 47
Public Perceptions of Resuming Elective Surgery During the COVID-19 Pandemic
Moverman, M. A., Puzzitiello, R. N., Pagani, N. R., Barnes, C. L., Jawa, A., & Menendez, M. E. (2020) · The Journal of Arthroplasty · doi:10.1016/j.arth.2020.07.037 · Cited by 37
The Effects of a Self-Similar Avatar Voice in Educational Games
Kao, D., Ratan, R., Mousas, C., & Magana, A. J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3474665 · Cited by 41

Authors: Chandler, J., & Shapiro, D.
Affiliations: Mathematica Policy Research (US)
University of Michigan (US)
ORCID: Chandler, Jesse
Venue: Annual Review of Clinical Psychology
Publisher: Annual Reviews
Published: 2016-03-28
Volume: 12
Issue: 1
Pages: 53-81
ISSN: 1548-5943 (ISSN-L)
1548-5951
DOI: 10.1146/annurev-clinpsy-021815-093623
OpenAlex: W2126493218
Cited by: 1,072
References: 177
Citations by year: 2026: 27 · 2025: 70 · 2024: 105 · 2023: 103 · 2022: 150 · 2021: 168 · 2020: 122 · 2019: 115 · 2018: 111 · 2017: 77 · 2016: 24
FWCI: 17.58
Type: journal article
Language: en
Open access: closed
Locations: Annual Review of Clinical Psychology (publishedVersion)
PubMed (publishedVersion)
RePEc: Research Papers in Economics (submittedVersion)
Topics: Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
Keywords: crowdsourcing, Amazon Mechanical Turk, convenience samples, non-probability sampling, sample composition, data quality, participant recruitment, digital labour markets
MeSH terms: Humans, Biomedical Research, Crowdsourcing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Conducting Clinical Research Using Crowdsourced Convenience Samples

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks