Cited by 179Mobile Crowdsensing and CrowdsourcingPersonality Disorders and PsychopathologyPersonality Traits and Psychology

Using online, crowdsourcing platforms for data collection in personality disorder research: the example of Amazon’s Mechanical Turk

Miller, Joshua D., Crowe, Michael, Weiss, Brandon, Maples-Keller, Jessica L., Lynam, Donald R.·2017·Personality Disorders Theory Research and Treatment

Abstract

The use of crowdsourcing platforms such as Amazon's Mechanical Turk (MTurk) for data collection in the behavioral sciences has increased substantially in the past several years due in large part to (a) the ability to recruit large samples, (b) the inexpensiveness of data collection, (c) the speed of data collection, and (d) evidence that the data collected are, for the most part, of equal or better quality to that collected in undergraduate research pools. In this review, we first evaluate the strengths and potential limitations of this approach to data collection. Second, we examine how MTurk has been used to date in personality disorder (PD) research and compare the characteristics of such research to PD research conducted in other settings. Third, we compare PD trait data from the Section III trait model of the DSM-5 collected via MTurk to data collected using undergraduate and clinical samples with regard to internal consistency, mean-level differences, and factor structure. Overall, we conclude that platforms such as MTurk have much to offer PD researchers, especially for certain kinds of research (e.g., where large samples are required and there is a need for iterative sampling). Whether MTurk itself remains the predominant model of such platforms is unclear, however, and will largely depend on decisions related to cost effectiveness and the development of alternatives that offer even greater flexibility. (PsycINFO Database Record

Integrity signals

Registry

Crossref · 10.1037/per0000191

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Data gathered using MTurk are high-quality, with high levels of internal consistency compared to data collected using different methods.

    “Additionally, the data gathered using MTurk are high-quality, with high levels of internal consistency compared to data collected using different methods [61].”
  • Data collected via MTurk is considered equal or better quality compared to data collected from undergraduate research pools.

    “Compared to data collected from undergraduate research pools, the data collected via MTurk is considered equal or better quality (Miller et al., 2017).”

Cites

Flagged first, then by citations

1 of the 44 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Miller, J. D., Crowe, M., Weiss, B., Maples-Keller, J. L., & Lynam, D. R.
Affiliations
  • University of Georgia (US)
  • Purdue University West Lafayette (US)
Venue
Personality Disorders Theory Research and Treatment
Published
2017
Volume
8
Issue
1
Pages
26-34
ISSN
  • 1949-2715 (ISSN-L)
  • 1949-2723
OpenAlex
W2565779926
Cited by
179
References
44
Citations by year
2026: 2 · 2025: 6 · 2024: 12 · 2023: 22 · 2022: 13 · 2021: 25 · 2020: 21 · 2019: 37 · 2018: 27 · 2017: 13 · 2016: 1
FWCI
32.19
Type
journal article
Language
en
Open access
closed
Locations
  • Personality Disorders Theory Research and Treatment (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
  • Social Sciences › Psychology › Clinical Psychology › Personality Disorders and Psychopathology
  • Social Sciences › Psychology › Clinical Psychology › Personality Traits and Psychology
Keywords
Amazon Mechanical Turk, crowdsourcing, online data collection, personality disorders, DSM-5, trait data, sample recruitment, data quality, factor structure, internal consistency, cost-effectiveness
MeSH terms
Data Collection, Humans, Personality Disorders, Internet, Biomedical Research, Crowdsourcing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review