The Turker Blues: Hidden Factors Behind Increased Depression Rates Among Amazon’s Mechanical Turkers
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Data collection from online platforms, such as Amazon’s Mechanical Turk (MTurk), has become popular in clinical research. However, there are also concerns about the representativeness and the quality of these data for clinical studies. The present work explores these issues in the specific case of major depression. Analyses of two large data sets gathered from MTurk (Sample 1: N = 2,692; Sample 2: N = 2,354) revealed two major findings: First, failing to screen for inattentive and fake respondents inflates the rates of major depression artificially and significantly (by 18.5%–27.5%). Second, after cleaning the data sets, depression in MTurk is still 1.6 to 3.6 times higher than general population estimates. Approximately half of this difference can be attributed to differences in the composition of MTurk samples and the general population (i.e., sociodemographics, health, and physical activity lifestyle). Several explanations for the other half are proposed, and practical data-quality tools are provided.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.10.1092 · Cited by 31,765
Spitzer, R. L. (1999) · JAMA · doi:10.1001/jama.282.18.1737 · Cited by 9,802
Beck, A. T., Steer, R. A., & Brown, G. (2011) · PsycTESTS Dataset · doi:10.1037/t00742-000 · Cited by 7,535
UCLA loneliness scale (version 3): reliability, validity, and factor structure
Russell, D. W. (1996) · Journal of Personality Assessment · doi:10.1207/s15327752jpa6601_2 · Cited by 5,800
Nolen-Hoeksema, S., Wisco, B. E., & Lyubomirsky, S. (2008) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2008.00088.x · Cited by 5,172
Nolen-Hoeksema, S., & Morrow, J. (1991) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.61.1.115 · Cited by 3,276
Sensitive questions in surveys.
Tourangeau, R., & Yan, T. (2007) · Psychological Bulletin · doi:10.1037/0033-2909.133.5.859 · Cited by 3,057 · cited 1×
“The stigma could inhibit depressed individuals from sharing personal information in face-to-face interactions (de Leeuw, 1992; Tourangeau & Yan, 2007), which is the standard method of data collection in the NHANES.”
MethodKessler, R. C., Petukhova, M., Sampson, N. A., Zaslavsky, A. M., & Wittchen, H. (2012) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1359 · Cited by 2,947
Cacioppo, J. T., Hughes, M. E., Waite, L. J., Hawkley, L. C., & Thisted, R. A. (2006) · Psychology and Aging · doi:10.1037/0882-7974.21.1.140 · Cited by 2,380 · cited 1×
“It is possible that individuals who work for online data collection forums for extended periods during a day are at increased risk of developing feelings of loneliness, which is a well-documented risk factor for developing depressive symptoms (Cacioppo et al., 2006).”
Background
Cited by
Data Collection via Online Platforms: Challenges and Recommendations for Future Research
Newman, A., Bavik, Y. L., Mount, M., & Shao, B. (2021) · Applied Psychology · doi:10.1111/apps.12302 · Cited by 306
Kujawa, A., Green, H., Compas, B. E., Dickey, L., & Pegg, S. (2020) · Depression and Anxiety · doi:10.1002/da.23109 · Cited by 278
Park, C. L., Finkelstein-Fox, L., Russell, B. S., Fendrich, M., Hutchison, M., & Becker, J. (2021) · American Psychologist · doi:10.1037/amp0000813 · Cited by 127
Improving Data Quality Using Amazon Mechanical Turk Through Platform Setup
Lu, L., Neale, N., Line, N. D., & Bonn, M. (2022) · Cornell Hospitality Quarterly · doi:10.1177/19389655211025475 · Cited by 94
“Workers with suspicious IP addresses are also much less likely to pass attention checks (e.g., Ophir et al., 2020; S. M. Smith et al., 2016).”
BackgroundMethodWorker Health and Well-Being in the Gig Economy: A Proposed Framework and Research Agenda
Keith, M. G., Harms, P. D., & Long, A. C. (2020) · Entrepreneurial and Small Business Stressors, Experienced Stress, and Well-Being · doi:10.1108/s1479-355520200000018002 · Cited by 72
Metadata
- Authors
- Ophir, Y., Sisso, I., Asterhan, C. S. C., Tikochinski, R., & Reichart, R.
- Venue
- Clinical Psychological Science
- Publisher
- SAGE Publications
- Published
- 2019-10-02
- Volume
- 8
- Issue
- 1
- Pages
- 65-83
- ISSN
- 2167-7026 (ISSN-L)
- 2167-7034
- OpenAlex
- W2978801451
- Cited by
- 126
- References
- 32
- FWCI
- 20.42
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Turker Blues: Hidden Factors Behind Increased Depression Rates Among Amazon’s Mechanical Turkers. https://app.cochat.ai/papers/10.1177/2167702619865973
- The paper
- Ophir, Y., Sisso, I., Asterhan, C. S. C., Tikochinski, R., & Reichart, R. (2020). The Turker Blues: Hidden Factors Behind Increased Depression Rates Among Amazon’s Mechanical Turkers. Clinical Psychological Science. https://doi.org/10.1177/2167702619865973
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.