Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.

A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk

Open access
Hara, Kotaro, Adams, Abigail, Milland, Kristy, Savage, Saiph, Callison-Burch, Chris, Bigham, Jeffrey P.·2018·Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Abstract

A growing number of people are working as part of on-line crowd work. Crowd work is often thought to be low wage work. However, we know little about the wage distribution in practice and what causes low/high earnings in this setting. We recorded 2,676 workers performing 3.8 million tasks on Amazon Mechanical Turk. Our task-level analysis revealed that workers earned a median hourly wage of only ~$2/h, and only 4% earned more than $7.25/h. While the average requester pays more than $11/h, lower-paying requesters post much more work. Our wage calculations are influenced by how unpaid work is accounted for, e.g., time spent searching for tasks, working on tasks that are rejected, and working on tasks that are ultimately not submitted. We further explore the characteristics of tasks and working patterns that yield higher hourly wages. Our analysis informs platform design and worker tools to create a more positive future for crowd work.

Overview

Integrity signals

Registry

Crossref · 10.1145/3173574.3174023

Cites

64 of 64 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 64 references

Research works printed but not linked: 9

  1. 10.1016/j.chb.2015.08.031 · DOI · Paper page
  2. 10.5555/1866696.1866697 · doi:10.5555/1866696.1866697
  3. 10.5555/2908698.2908703 · doi:10.5555/2908698.2908703
  4. 10.5555/1613715.1613763 · doi:10.5555/1613715.1613763
  5. 10.5555/540298 · doi:10.5555/540298
  6. 10.5555/2517747 · doi:10.5555/2517747
  7. Philippe Marcadent, Janine Berg, Mariya Aleksynska, Valerio De Stefano, Martine Humblet, Christina Behrendt, Susan Hayter, Christopher Land-Kazlauskas, Angelika Muller, Niall O'Higgins, Friederike Eberlein, Simone Marino, Bao Chau Le, and Calum Carson. 2016. NonStandard Employment Around The World.
  8. Juan Ramos and others. 2003. Using tf-idf to determine word relevance in document queries. In Proceedings of the first instructional conference on machine learning, 133--142.
  9. Amazon Mechanical Turk Participation Agreement (2014). Retrieved July 1, 2017 from https://www.mturk.com/mturk/conditionsofuse

Cited by

See all 554

Metadata

Authors
Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. P.
Affiliations
  • Carnegie Mellon University (US)
  • University of Oxford (GB)
  • McMaster University (CA)
  • West Virginia University (US)
  • University of Pennsylvania (US)
Venue
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2018-04-21
Pages
1-14
OpenAlex
W2777647957
Cited by
497
References
64
Citations by year
2026: 19 · 2025: 38 · 2024: 42 · 2023: 71 · 2022: 99 · 2021: 85 · 2020: 66 · 2019: 63 · 2018: 12
FWCI
66.64
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Funders
Economic and Social Research Council (Grant ES/N017099/1)
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
  • Social Sciences › Sociology and Political Science › Digital Economy and Work Transformation
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Labor market dynamics and wage inequality
Keywords
crowdwork, digital labour markets, worker income, wage distribution, Amazon Mechanical Turk, task pricing, unpaid work, task rejection, platform design, low-wage employment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk. https://app.cochat.ai/papers/10.1145/3173574.3174023
The paper
Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. P. (2018). A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3173574.3174023

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.