CoChat

“Who are these people?” Evaluating the demographic characteristics and political preferences of MTurk survey respondents

Open access
Huff, Connor, Tingley, Dustin · 2015 · Research & Politics

The DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.

Abstract

As Amazon’s Mechanical Turk (MTurk) has surged in popularity throughout political science, scholars have increasingly challenged the external validity of inferences made drawing upon MTurk samples. At workshops and conferences experimental and survey-based researchers hear questions about the demographic characteristics, political preferences, occupation, and geographic location of MTurk respondents. In this paper we answer these questions and present a number of novel results. By introducing a new benchmark comparison for MTurk surveys, the Cooperative Congressional Election Survey, we compare the joint distributions of age, gender, and race among MTurk respondents within the United States. In addition, we compare political, occupational, and geographical information about respondents from MTurk and CCES. Throughout the paper we show several ways that political scientists can use the strengths of MTurk to attract respondents with specific characteristics of interest to best answer their substantive research questions.

Overview

Integrity signals

Registry

Crossref · 10.1177/2053168015604648

Cites

24 of 24 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Research works printed but not linked: 3

  1. Gay C, 2015, Journal of Race, Ethnicity, and Politics
  2. Leeper T, 2013, The Political Methodologist, 20, 2
  3. Leeper TJ, 2015, 6, 5, 1

Cited by

See all 961

Metadata

Authors
Huff, C., & Tingley, D.
Affiliations
Harvard University (US)
Venue
Research & Politics
Published
2015-07-01
Volume
2
Issue
3
ISSN
2053-1680 (ISSN-L)
OpenAlex
W1936420698
Cited by
903
References
25
Citations by year
2026: 25 · 2025: 41 · 2024: 53 · 2023: 71 · 2022: 111 · 2021: 156 · 2020: 112 · 2019: 106 · 2018: 107 · 2017: 81 · 2016: 34 · 2015: 2
FWCI
112.28
Type
journal article
Language
en
Open access
Licence
  • cc-by
  • journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
  • Research & Politics (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Institute for Quantitative Social Science, Harvard University
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
  • Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
  • Social Sciences › Political Science and International Relations › Electoral Systems and Political Participation
Keywords
Mechanical Turk, generalizability, respondent demographics, political preferences, job characteristics, geographic location

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.