The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
The importance of assessing clinical phenomena in Mechanical Turk research.
Open accessAbstract
Amazon.com's Mechanical Turk (MTurk) website provides a data collection platform with quick and inexpensive access to diverse samples. Numerous reports have lauded MTurk as capturing high-quality data with an epidemiological sample that is more representative of the U.S. population than traditional in-person convenience samples (e.g., undergraduate subject pools). This benefit, in combination with the ease and low-cost of data collection, has led to a remarkable increase in studies using MTurk to investigate phenomena across a wide range of psychological disciplines. Multiple reports have now examined the demographic characteristics of MTurk samples. One key gap remains, however, in that relatively little is known about individual differences in clinical symptoms among MTurk participants. This paper discusses the importance of assessing clinical phenomena in MTurk samples and supports its assertions through an empirical investigation of a large sample (N = 1,098) of MTurk participants. Results revealed that MTurk participants endorse clinical symptoms to a substantially greater degree than traditional nonclinical samples. This distinction was most striking for depression and social anxiety symptoms, which were endorsed at levels comparable with individuals with clinically diagnosed mood and anxiety symptoms. Participants' symptoms of physiological anxiety, hoarding, and eating pathology fell within the subclinical range. Overall, the number of individuals exceeding validated clinical cutoffs was between 3 and 19 times the estimated 12-month prevalence rates. Based on the current findings, it is argued that MTurk participants differ from the general population in meaningful ways, and researchers should consider this when referring to this sample as truly representative. (PsycINFO Database Record
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Lovibond, P. F., & Lovibond, S. H. (1995) · Behaviour Research and Therapy · doi:10.1016/0005-7967(94)00075-u · Cited by 14,523
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Henry, J. D., & Crawford, J. R. (2005) · British Journal of Clinical Psychology · doi:10.1348/014466505x29657 · Cited by 5,801
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Rumination reconsidered: a psychometric analysis.
Treynor, W., Gonzalez, R., & Nolen-Hoeksema, S. (2003) · Cognitive Therapy and Research · doi:10.1023/a:1023910315561 · Cited by 3,808
Mattick, R. P., & Clarke, J. C. (1998) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(97)10031-6 · Cited by 3,367
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004) · American Psychologist · doi:10.1037/0003-066x.59.2.93 · Cited by 2,932
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Taylor, S., Zvolensky, M. J., Cox, B. J., Deacon, B., Heimberg, R. G., Ledley, D. R., Abramowitz, J. S., Holaway, R. M., Sandin, B., Stewart, S. H., et al. (2007) · Psychological Assessment · doi:10.1037/1040-3590.19.2.176 · Cited by 1,866
Psychometric properties of the Depression Anxiety Stress Scales (DASS) in clinical samples
Brown, T. A., Chorpita, B. F., Korotitsch, W., & Barlow, D. H. (1997) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(96)00068-x · Cited by 1,860
The Distress Tolerance Scale: Development and Validation of a Self-Report Measure
Simons, J. S., & Gaher, R. M. (2005) · Motivation and Emotion · doi:10.1007/s11031-005-7955-3 · Cited by 1,730
Casler, K., Bickel, L., & Hackett, E. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.05.009 · Cited by 1,662
Using Mechanical Turk to Study Clinical Populations
Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013) · Clinical Psychological Science · doi:10.1177/2167702612469015 · Cited by 1,139
Osman, A., Wong, J. L., Bagge, C. L., Freedenthal, S., Gutierrez, P. M., & Lozano, G. (2012) · Journal of Clinical Psychology · doi:10.1002/jclp.21908 · Cited by 1,064
The viability of crowdsourcing for survey research
Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0081-0 · Cited by 1,018
Analyzing the Amazon Mechanical Turk marketplace
Ipeirotis, P. G. (2010) · XRDS: Crossroads, The ACM Magazine for Students · doi:10.1145/1869086.1869094 · Cited by 879
Leyro, T. M., Zvolensky, M. J., & Bernstein, A. (2010) · Psychological Bulletin · doi:10.1037/a0019712 · Cited by 868
Abramowitz, J. S., Deacon, B. J., Olatunji, B. O., Wheaton, M. G., Berman, N. C., Losardo, D., Timpano, K. R., McGrath, P. B., Riemann, B. C., Adams, T., et al. (2010) · Psychological Assessment · doi:10.1037/a0018260 · Cited by 848
Self-citationIncreasingly certain about uncertainty: Intolerance of uncertainty across anxiety and depression
Carleton, R. N., Mulvogue, M. K., Thibodeau, M. A., McCabe, R. E., Antony, M. M., & Asmundson, G. J. G. (2012) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2012.01.011 · Cited by 775
Widiger, T. A., & Samuel, D. B. (2005) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.114.4.494 · Cited by 752
JORM, A. F. (2000) · Psychological Medicine · doi:10.1017/s0033291799001452 · Cited by 655
Lifetime risk and persistence of psychiatric disorders across ethnic groups in the United States
BRESLAU, J., KENDLER, K. S., SU, M., GAXIOLA-AGUILAR, S., & KESSLER, R. C. (2005) · Psychological Medicine · doi:10.1017/s0033291704003514 · Cited by 642
Neurocognitive impairment in drug-free patients with major depressive disorder
Porter, R. J., Gallagher, P., Thompson, J. M., & Young, A. H. (2003) · British Journal of Psychiatry · doi:10.1192/bjp.182.3.214 · Cited by 519
Compulsive hoarding: current status of the research
Steketee, G., & Frost, R. (2003) · Clinical Psychology Review · doi:10.1016/j.cpr.2003.08.002 · Cited by 498
Anxiety sensitivity: Prospective prediction of panic attacks and Axis I pathology
Schmidt, N. B., Zvolensky, M. J., & Maner, J. K. (2006) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2006.07.009 · Cited by 485
Buhr, K., & Dugas, M. J. (2006) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2004.12.004 · Cited by 421
A brief interview for assessing compulsive hoarding: The Hoarding Rating Scale-Interview
Tolin, D. F., Frost, R. O., & Steketee, G. (2010) · Psychiatry Research · doi:10.1016/j.psychres.2009.05.001 · Cited by 408
Anxiety sensitivity and intolerance of uncertainty: requisites of the fundamental fears?
Nicholas Carleton, R., Sharpe, D., & Asmundson, G. J. G. (2007) · Behaviour Research and Therapy · doi:10.1016/j.brat.2007.04.006 · Cited by 293
Peters, L. (2000) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(99)00131-x · Cited by 252
Dimensions of anxiety sensitivity in the anxiety disorders: evaluation of the ASI-3
Wheaton, M. G., Deacon, B. J., McGrath, P. B., Berman, N. C., & Abramowitz, J. S. (2012) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2012.01.002 · Cited by 245
Joormann, J., Levens, S. M., & Gotlib, I. H. (2011) · Psychological Science · doi:10.1177/0956797611415539 · Cited by 209
Timpano, K. R., Exner, C., Glaesmer, H., Rief, W., Keshaviah, A., Brähler, E., & Wilhelm, S. (2011) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.10m06380 · Cited by 196
Self-citationYip, J. A., & Côté, S. (2012) · Psychological Science · doi:10.1177/0956797612450031 · Cited by 166
Depression and rumination: Relation to components of inhibition
Zetsche, U., D'Avanzato, C., & Joormann, J. (2012) · Cognition & Emotion · doi:10.1080/02699931.2011.613919 · Cited by 134
Social Psychological Methods Outside the Laboratory
Reis, H. T., & Gosling, S. D. (2010) · Wiley eBooks · doi:10.1002/9780470561119.socpsy001003 · Cited by 110
Mitchell, M. A., Riccardi, C. J., Keough, M. E., Timpano, K. R., & Schmidt, N. B. (2013) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2012.12.003 · Cited by 62
Self-citationGarner (1983) · doi:10.1002/1098-108x(198321)2:2<15::aid-eat2260020203>3.0.co;2-6
Cited by
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072
An Evaluation of Amazon’s Mechanical Turk, Its Rapid Rise, and Its Effective Use
Buhrmester, M. D., Talaifar, S., & Gosling, S. D. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691617706516 · Cited by 767
Detecting depression and mental illness on social media: an integrative review
Guntuku, S. C., Yaden, D. B., Kern, M. L., Ungar, L. H., & Eichstaedt, J. C. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.07.005 · Cited by 722
Crowdsourcing Consumer Research
Goodman, J. K., & Paolacci, G. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx047 · Cited by 588
The development and initial validation of the Decent Work Scale.
Duffy, R. D., Allan, B. A., England, J. W., Blustein, D. L., Autin, K. L., Douglass, R. P., Ferreira, J., & Santos, E. J. R. (2017) · Journal of Counseling Psychology · doi:10.1037/cou0000191 · Cited by 505
Metadata
- Authors
- Arditte, K. A., Çek, D., Shaw, A. M., & Timpano, K. R.
- Venue
- Psychological Assessment
- Published
- 2016-06
- Volume
- 28
- Issue
- 6
- Pages
- 684-691
- ISSN
- 1040-3590 (ISSN-L)
- 1939-134X
- OpenAlex
- W1695897010
- Cited by
- 311
- References
- 38
- FWCI
- 32.67
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.