The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
The importance of assessing clinical phenomena in Mechanical Turk research.
Open accessAbstract
Amazon.com's Mechanical Turk (MTurk) website provides a data collection platform with quick and inexpensive access to diverse samples. Numerous reports have lauded MTurk as capturing high-quality data with an epidemiological sample that is more representative of the U.S. population than traditional in-person convenience samples (e.g., undergraduate subject pools). This benefit, in combination with the ease and low-cost of data collection, has led to a remarkable increase in studies using MTurk to investigate phenomena across a wide range of psychological disciplines. Multiple reports have now examined the demographic characteristics of MTurk samples. One key gap remains, however, in that relatively little is known about individual differences in clinical symptoms among MTurk participants. This paper discusses the importance of assessing clinical phenomena in MTurk samples and supports its assertions through an empirical investigation of a large sample (N = 1,098) of MTurk participants. Results revealed that MTurk participants endorse clinical symptoms to a substantially greater degree than traditional nonclinical samples. This distinction was most striking for depression and social anxiety symptoms, which were endorsed at levels comparable with individuals with clinically diagnosed mood and anxiety symptoms. Participants' symptoms of physiological anxiety, hoarding, and eating pathology fell within the subclinical range. Overall, the number of individuals exceeding validated clinical cutoffs was between 3 and 19 times the estimated 12-month prevalence rates. Based on the current findings, it is argued that MTurk participants differ from the general population in meaningful ways, and researchers should consider this when referring to this sample as truly representative. (PsycINFO Database Record
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Lovibond, P. F., & Lovibond, S. H. (1995) · Behaviour Research and Therapy · doi:10.1016/0005-7967(94)00075-u · Cited by 14,523
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Henry, J. D., & Crawford, J. R. (2005) · British Journal of Clinical Psychology · doi:10.1348/014466505x29657 · Cited by 5,801
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Rumination reconsidered: a psychometric analysis.
Treynor, W., Gonzalez, R., & Nolen-Hoeksema, S. (2003) · Cognitive Therapy and Research · doi:10.1023/a:1023910315561 · Cited by 3,808
Mattick, R. P., & Clarke, J. C. (1998) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(97)10031-6 · Cited by 3,367
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004) · American Psychologist · doi:10.1037/0003-066x.59.2.93 · Cited by 2,932
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Taylor, S., Zvolensky, M. J., Cox, B. J., Deacon, B., Heimberg, R. G., Ledley, D. R., Abramowitz, J. S., Holaway, R. M., Sandin, B., Stewart, S. H., et al. (2007) · Psychological Assessment · doi:10.1037/1040-3590.19.2.176 · Cited by 1,866
Cited by
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072
An Evaluation of Amazon’s Mechanical Turk, Its Rapid Rise, and Its Effective Use
Buhrmester, M. D., Talaifar, S., & Gosling, S. D. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691617706516 · Cited by 767
Detecting depression and mental illness on social media: an integrative review
Guntuku, S. C., Yaden, D. B., Kern, M. L., Ungar, L. H., & Eichstaedt, J. C. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.07.005 · Cited by 722
Crowdsourcing Consumer Research
Goodman, J. K., & Paolacci, G. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx047 · Cited by 588
The development and initial validation of the Decent Work Scale.
Duffy, R. D., Allan, B. A., England, J. W., Blustein, D. L., Autin, K. L., Douglass, R. P., Ferreira, J., & Santos, E. J. R. (2017) · Journal of Counseling Psychology · doi:10.1037/cou0000191 · Cited by 505
Mortensen, K., & Hughes, T. L. (2018) · Journal of General Internal Medicine · doi:10.1007/s11606-017-4246-0 · Cited by 441
Crowdsourcing research: Data collection with Amazon’s Mechanical Turk
Sheehan, K. B. (2018) · Communication Monographs · doi:10.1080/03637751.2017.1342043 · Cited by 408
Measuring well-being: A comparison of subjective well-being and PERMA
Goodman, F. R., Disabato, D. J., Kashdan, T. B., & Kauffman, S. B. (2018) · The Journal of Positive Psychology · doi:10.1080/17439760.2017.1388434 · Cited by 367
Mental distress among U.S. adults during the COVID-19 pandemic
Twenge, J. M., & Joiner, T. E. (2020) · Journal of Clinical Psychology · doi:10.1002/jclp.23064 · Cited by 330
“Abbreviations: CI, confidence interval; d, difference in standard deviations; K6, Kessler‐6 scale; NHIS, National Health Interview Survey. three times (Arditte et al., 2016); even if that were the case here, serious mental distress would have still nearly tripled from before to during the pandemic.”
BackgroundKeith, M. G., Tay, L., & Harms, P. D. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01359 · Cited by 319
Kujawa, A., Green, H., Compas, B. E., Dickey, L., & Pegg, S. (2020) · Depression and Anxiety · doi:10.1002/da.23109 · Cited by 278
Miller, J. D., Crowe, M., Weiss, B., Maples-Keller, J. L., & Lynam, D. R. (2017) · Personality Disorders Theory Research and Treatment · doi:10.1037/per0000191 · Cited by 179
Living “in the zone”: hyperfocus in adult ADHD
Hupfeld, K. E., Abagis, T. R., & Shah, P. (2019) · ADHD Attention Deficit and Hyperactivity Disorders · doi:10.1007/s12402-018-0272-y · Cited by 157
Park, C. L., Finkelstein-Fox, L., Russell, B. S., Fendrich, M., Hutchison, M., & Becker, J. (2021) · American Psychologist · doi:10.1037/amp0000813 · Cited by 127
Holte, A. J., & Ferraro, F. R. (2020) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106465 · Cited by 122
On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?
Dziri, N., Milton, S., Yu, M., Zaiane, O., & Reddy, S. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.387 · Cited by 121
Rogers, M. L., & Joiner, T. E. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2017.12.017 · Cited by 86
Development and evaluation of the Internalized Racism in Asian Americans Scale (IRAAS)
Choi, A. Y., Israel, T., & Maeda, H. (2017) · Journal of Counseling Psychology · doi:10.1037/cou0000183 · Cited by 70
Yap, K., & Grisham, J. R. (2019) · Journal of Behavioral Addictions · doi:10.1556/2006.8.2019.15 · Cited by 71
Safety behaviors, experiential avoidance, and anxiety: A path analysis approach
Kirk, A., Meyer, J. M., Whisman, M. A., Deacon, B. J., & Arch, J. J. (2019) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2019.03.002 · Cited by 59
Nelson, G. H., O'Hara, M. W., & Watson, D. (2018) · Journal of Clinical Psychology · doi:10.1002/jclp.22560 · Cited by 51
Mental distress among U.S. adults during the COVID-19 pandemic
Twenge, J., & Joiner, T. (2020) · doi:10.31234/osf.io/wc8ud · Cited by 330
Boughner, E., Thornley, E., Kharlas, D., & Frewen, P. (2016) · Mindfulness · doi:10.1007/s12671-016-0502-3 · Cited by 51
Public knowledge of late-life cognitive decline and dementia in an international sample
Van Patten, R., & Tremont, G. (2020) · Dementia · doi:10.1177/1471301218805923 · Cited by 40
Beshai, S., Bueno, C., Yu, M., Feeney, J. R., & Pitariu, A. (2020) · Behaviour Research and Therapy · doi:10.1016/j.brat.2020.103724 · Cited by 43
Body dysmorphic disorder symptoms and risk for suicide: The role of depression
Shaw, A. M., Arditte Hall, K. A., Rosenfield, E., & Timpano, K. R. (2016) · Body Image · doi:10.1016/j.bodyim.2016.09.007 · Cited by 29
Self-citationObsessive-Compulsive and Depressive Symptoms: The Role of Depressive Cognitive Styles
Shaw, A. M., Carbonella, J. Y., Arditte Hall, K. A., & Timpano, K. R. (2017) · The Journal of Psychology · doi:10.1080/00223980.2017.1372341 · Cited by 17
Self-citationChampion, S. M., & Grisham, J. R. (2022) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2021.101720 · Cited by 10
Earleywine, M., Altman, B. R., & De Leo, J. (2022) · Journal of Psychoactive Drugs · doi:10.1080/02791072.2021.1912863 · Cited by 9
(2018) · Digital Commons - University of South Florida (University of South Florida) · Cited by 4
Metadata
- Authors
- Arditte, K. A., Çek, D., Shaw, A. M., & Timpano, K. R.
- Venue
- Psychological Assessment
- Published
- 2016-06
- Volume
- 28
- Issue
- 6
- Pages
- 684-691
- ISSN
- 1040-3590 (ISSN-L)
- 1939-134X
- OpenAlex
- W1695897010
- Cited by
- 311
- References
- 38
- FWCI
- 32.67
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.