A typology of artificial intelligence data work
Open accessAbstract
This article provides a new typology for understanding human labour integrated into the production of artificial intelligence systems through data preparation and model evaluation. We call these forms of labour ‘AI data work’ and show how they are an important and necessary element of the artificial intelligence production process. We draw on fieldwork with an artificial intelligence data business process outsourcing centre specialising in computer vision data, alongside a decade of fieldwork with microwork platforms, business process outsourcing, and artificial intelligence companies to help dispel confusion around the multiple concepts and frames that encompass artificial intelligence data work including ‘ghost work’, ‘microwork’, ‘crowdwork’ and ‘cloudwork’. We argue that these different frames of reference obscure important differences between how this labour is organised in different contexts. The article provides a conceptual division between the different types of artificial intelligence data work institutions and the different stages of what we call the artificial intelligence data pipeline. This article thus contributes to our understanding of how the practices of workers become a valuable commodity integrated into global artificial intelligence production networks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The AI industry relies heavily on human workers who clean, tag, classify, and verify data and model outputs.
“Behind the illusions of autonomy and machine intelligence are vast global networks of human workers who clean, tag, classify, and verify data and model outputs [96].”
From 'The plan is just survival': Data Work in Kenya and the Regime of Entrapment (2026) · cites this paper as [96]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 3× · see citing works
“These systems are designed with machine learning algorithms that leverage large amounts of data, energy and computational power (Bender et al., 2021; Crawford, 2021).”
Casler, K., Bickel, L., & Hackett, E. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.05.009 · Cited by 1,662 · see citing works
Tubaro, P., Casilli, A. A., & Coville, M. (2020) · Big Data & Society · doi:10.1177/2053951720919776 · Cited by 262 · see citing works
The platform as factory: Crowdwork and the hidden labour behind artificial intelligence
Altenried, M. (2020) · Capital & Class · doi:10.1177/0309816819899410 · Cited by 242 · see citing works
Micro-work, artificial intelligence and the automotive industry
Tubaro, P., & Casilli, A. A. (2019) · Journal of Industrial and Business Economics · doi:10.1007/s40812-019-00121-1 · Cited by 122 · see citing works
The poverty of ethical AI: impact sourcing and AI supply chains
Muldoon, J., Cant, C., Graham, M., & Ustek Spilda, F. (2025) · AI & SOCIETY · doi:10.1007/s00146-023-01824-9 · Cited by 70 · cited 5× · see citing works
“This project involved fi eldwork conducted in 2023 with a BPO specialising in AI data work in Kenya and Uganda (Muldoon et al., 2023).”
InfluentialSelf-citation
Flagged first, then by citations
Research works printed but not linked: 33
- Aloisi A and De Stefano V (2022) Your Boss Is an Algorithm Artificial Intelligence, Platform Work and Labour. London: Bloomsbury. Available at: https://onlinelibrary.wiley.com/ doi/abs/10.1111/bjir.12727 (accessed 23 February 2023). · DOI
- Anwar MA and Graham M (2022) The Digital Continent: Placing Africa in Planetary Networks of Work. Oxford: Oxford University Press.
- Bergvall-Kåreborn B and Howcroft D (2014) Amazon Mechanical Turk and the commodification of labour. New Technology, Work and Employment 29(3): 213-223.
- Bossen C, Pine K, Cabitza F, et al. (2019) Data work in health care: An introduction. Health Informatics Journal 25(3): 465-474.
- Bratton BH (2016) The Stack: On Software and Sovereignty. Cambridge, MA: MIT Press.
- Chen W-C, Suri S and Gray ML (2019) More than money: Correlation among worker demographics, motivations, and participation in online labor market. Proceedings of the International AAAI Conference on Web and Social Media 13: 134-145.
- Crawford K (2021) Atlas of AI: Power, Politics and the Planetary Costs of Artificial Intelligence. Yale and London: Yale University Press.
- Dauvergne P (2022) Is artificial intelligence greening global supply chains? Exposing the political economy of environmen- tal costs. Review of International Political Economy 29(3): 696-718.
- De Stefano V (2016) The rise of the "just-in time workforce": On demand work, crowdwork, and labor protection in the "gig economy". Comparative Labor law and Policy Journal 37(3): 461-471.
- Gebru T, Morgenstern J, Vecchione B, et al. (2021) Datasheets for Datasets. arXiv:1803.09010. arXiv. · arXiv
- Graham M, Hjorth I and Lehdonvirta V (2017) Digital labour and development: Impacts of global digital labour platforms and the gig economy on worker livelihoods. Transfer: European Review of Labour and Research 23(2): 135-162.
- Graham M, Woodcock J, Heeks R, et al. (2020) The fairwork foundation: Strategies for improving platform work in a global context. Geoforum; Journal of Physical, Human, and Regional Geosciences 112: 100-103.
- Howcroft D and Bergvall-Kåreborn B (2019) A typology of crowdwork platforms. Work, Employment and Society 33(1): 21-38.
- Irani L (2015b) The cultural work of microwork. New Media & Society 17(5): 720-739.
- Janah L (2017) Give Work: Reversing Poverty One Job at a Time. New York: Penguin Random House. Available at: https:// www.penguinrandomhouse.com/books/546305/give-work-by- leila-janah/ (accessed 9 May 2023).
- Lacity MC, Solomon S, Yan A, et al. (2011) Business process out- sourcing studies: A critical review and research directions. Journal of Information Technology 26(4): 221-258.
- Lu AJ, Dillahunt TR, Marcu G, et al. (2021) Data work in educa- tion: Enacting and negotiating care and control in teachers' use of data-driven classroom surveillance technology. Proceedings of the ACM on Human-Computer Interaction 5, CSCWS: 1-26. DOI/10.1145/3479596. · DOI
- Lubke G, Bhargava S, Valente J, et al. (2023) Fairwork Cloudwork Ratings 2023: Work in the Planetary Labour Market. Fairwork: Oxford Internet Institute. https://fair.work/ wp-content/uploads/sites/17/2023/07/Fairwork-Cloudwork- Ratings-2023-Red.pdf.
- Mehta A, Armenakis A, Mehta N, et al. (2006) Challenges and opportunities of business process outsourcing in India. Journal of Labor Research 27(3): 323-338.
- Miceli M and Posada J (2022) The data-production Dispositif. arXiv:2205.11963. arXiv. DOI: 10.48550/arXiv.2205.11963. · DOI
- Miceli M, Yang T, Alvarado Garcia A, et al. (2022) Documenting data production processes: A participatory approach for data work. Proceedings of the ACM on Human-Computer Interaction 6(CSCW2): 1-34.
- Mohamed S, Png M-T and Isaac W (2020) Decolonial AI: Decolonial theory as sociotechnical foresight in artificial intel- ligence. Philosophy & Technology 33(4): 659-684.
- Muldoon J and Wu BA (2023) Artificial intelligence in the colo- nial matrix of power. Philosophy & Technology 36(4): 1-24.
- North DC (1991) Institutions. Journal of Economic Perspectives 5(1): 97-112.
- Parmiggiani E, Østerlie T and Almklov PG (2022) In the back- rooms of data science. Journal of the Association for Information Systems 23(1): 139-164.
- Paullada A, Raji ID, Bender EM, et al. (2021) Data and its (dis) contents: A survey of dataset development and use in machine learning research. Patterns 2(11): 100336.
- Pine K and Bossen C (2020) Good organizational reasons for better medical records: The data work of clinical documenta- tion integrity specialists. Big Data & Society 7(2). doi.org/ 10.1177/2053951720965616. · DOI
- Posada J (2021) The coloniality of data work in Latin America. Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society 21 July 2021: 277-278.
- Schmidt F (2019) Crowdproduktion von trainingsdaten: Zur Rolle von Online-Arbeit beim Trainieren autonomer Fahrzeuge. Berlin: Hans-Böckler Stiftung. Available at: https://www. researchgate.net/publication/332964049.
- Schwarz D (2018) Embedded in the crowd: Creative freelancers, crowdsourced work, and occupational community. Work and Occupations 45(3): 247-282.
- Scott WR (2013) Institutions and Organizations: Ideas, Interests, and Identities, 4th ed. Los Angeles: Sage Publications.
- van Doorn N (2020) A new institution on the block: On platform urbanism and Airbnb citizenship. New Media & Society 22(10): 1808-1826.
- Weber M (1949) The Methodology of the Social Sciences. Free Press.
Cited by
Sorted by citations
Ramaul, L., Ritala, P., Kostis, A., & Aaltonen, P. (2026) · Journal of Management Studies · doi:10.1111/joms.13246 · Cited by 41 · see citing works
The Making of Performative Accuracy in AI Training: Precision Labor and Its Consequences
Zhang, B. Z., Yang, T., Miceli, M., Haimson, O. L., & Thomas, M. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713112 · Cited by 7 · see citing works
‘We can see a savage’: a case study of the colonial gaze in generative AI algorithms
Alenichev, A., Shaffer, J. D., Kingori, P., Grietens, K. P., Muldoon, J., & Rocher, L. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02685-0 · Cited by 3 · see citing works
“…capitalism, and lingering colonial patterns of power continue to shape the production of AI systems, notably in computer vision, research on how AI systems process, analyze, and understand digital images (Kwet 2019; Mohamed et al. 2020; Muldoon et al. 2024; Pasquinelli 2023; Gebru and Denton 2024).”
Self-citation'The plan is just survival': Data Work in Kenya and the Regime of Entrapment
Kapania, S., Yang, T., Abdelkadir, N. A., Scheuerman, M. K., Miceli, M., Taylor, A. S., & Fox, S. E. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791097 · Cited by 2 · see citing works
“Behind the illusions of autonomy and machine intelligence are vast global networks of human workers who clean, tag, classify, and verify data and model outputs [96].”
- Authors
- Muldoon, J., Cant, C., Wu, B., & Graham, M.
- Affiliations
- University of Essex (GB)
- Oxford Internet Institute, University of Oxford, UK
- University of Oxford (GB)
- Corresponding author
- Muldoon, James
- Venue
- Big Data & Society
- Publisher
- SAGE Publications
- Published
- 2024-03
- Volume
- 11
- Issue
- 1
- ISSN
- 2053-9517 (ISSN-L)
- OpenAlex
- W4392932113
- Cited by
- 50
- References
- 6
- Citations by year
- 2026: 18 · 2025: 23 · 2024: 8 · 2023: 1
- FWCI
- 35.02
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Big Data & Society (publishedVersion, cc-by)
- Open Access at Essex (University of Essex) (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- European Commission (2007–2013, FP/2007/2013)
- Bundesministerium für Wirtschaftliche Zusammenarbeit und Entwicklung
- Deutsche Gesellschaft für Internationale Zusammenarbeit
- Economic and Social Research Council (ES/S00081X/1)
- FP7 Ideas: European Research Council (335716)
- FP7 International Cooperation
- Topics
- Social Sciences › Sociology and Political Science › Digital Economy and Work Transformation
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Business, Management and Accounting › Industrial relations › Legal and Labor Studies
- Keywords
- data preparation, model evaluation, microwork, crowdwork, business process outsourcing, typology, labor organization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
