The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
The trainer, the verifier, the imitator: Three ways in which human platform workers support artificial intelligence
Open accessAbstract
This paper sheds light on the role of digital platform labour in the development of today’s artificial intelligence, predicated on data-intensive machine learning algorithms. Focus is on the specific ways in which outsourcing of data tasks to myriad ‘micro-workers’, recruited and managed through specialized platforms, powers virtual assistants, self-driving vehicles and connected objects. Using qualitative data from multiple sources, we show that micro-work performs a variety of functions, between three poles that we label, respectively, ‘artificial intelligence preparation’, ‘artificial intelligence verification’ and ‘artificial intelligence impersonation’. Because of the wide scope of application of micro-work, it is a structural component of contemporary artificial intelligence production processes – not an ephemeral form of support that may vanish once the technology reaches maturity stage. Through the lens of micro-work, we prefigure the policy implications of a future in which data technologies do not replace human workforce but imply its marginalization and precariousness.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As sociologists of technological innovation highlight, a lot of human labour is needed to fill the cracks of AI (Gray and Suri 2019;Tubaro and Casilli 2019;Tubaro et al. 2020).”
From Artificial intelligence and work: a critical review of recent research from the social sciences (2024) · cites this paper as Tubaro et al. 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1993) · Journal of the American Dietetic Association · doi:10.1016/0002-8223(93)92196-5 · Cited by 12,205
The future of employment: How susceptible are jobs to computerisation?
Frey, C. B., & Osborne, M. A. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.08.019 · Cited by 8,727
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1×
“While useful to assess the size and growth rate of the whole online (or cloud-based) global labour market (K€assi and Lehdonvirta, 2018), this approach obscures the linkages with data and AI production.”
BackgroundWhy Are There Still So Many Jobs? The History and Future of Workplace Automation
Autor, D. H. (2015) · Journal of Economic Perspectives · doi:10.1257/jep.29.3.3 · Cited by 3,917 · cited 1×
“…(Acemoglu and Restrepo, 2018), other studies highlight historical dynamics of complementarity rather than substitution between human labour and machinery (Autor, 2015; Bessen, 2017), with more complex outcomes such as polarization between high-and low-skilled workers (Autor and Dorn, 2013).”
BackgroundThe growth of low-skill service jobs and the polarization of the US labor market
Autor, D. H., & Dorn, D. (2013) · American Economic Review · doi:10.1257/aer.103.5.1553 · Cited by 3,824
Acemoglu, D., & Restrepo, P. (2018) · American Economic Review · doi:10.1257/aer.20160696 · Cited by 3,121 · cited 1×
“While some scientific research corroborates these predictions (Acemoglu and Restrepo, 2018), other studies highlight historical dynamics of complementarity rather than substitution between human labour and machinery (Autor, 2015; Bessen, 2017), with more complex outcomes such as polarization between…”
ResultThe Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies
Dong, X., & McIntyre, S. H. (2014) · Quantitative Finance · doi:10.1080/14697688.2014.946440 · Cited by 1,759
The Data Revolution: Big Data, Open Data, Data Infrastructures & Their Consequences
Kitchin, R. (2014) · doi:10.4135/9781473909472 · Cited by 1,469 · cited 1×
“As part of a broader drive to accumulate and leverage data resources (Kitchin, 2014), AI production needs human help not only to design cutting-edge algorithms (highly qualified engineers and computer scientists) but also at a much more basic level, to produce, enrich and curate data.”
Background(2019) · Cited by 1,082
An Introduction to Machine Learning
Kubát, M. (2015) · Springer eBooks · doi:10.1007/978-3-319-20010-1 · Cited by 1,024
The master algorithm: how the quest for the ultimate learning machine will remake our world
(2016) · Choice Reviews Online · doi:10.5860/choice.194685 · Cited by 810
Online labour index: Measuring the online gig economy for policy and research
kässi, O., & Lehdonvirta, V. (2018) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2018.07.056 · Cited by 591
The cultural work of microwork
Irani, L. (2015) · New Media & Society · doi:10.1177/1461444813511926 · Cited by 568
Roberts, S. T. (2019) · Yale University Press eBooks · doi:10.12987/9780300245318 · Cited by 559
Adams‐Prassl, J. (2018) · doi:10.1093/oso/9780198797012.001.0001 · Cited by 441
Difference and Dependence among Digital Workers: The Case of Amazon Mechanical Turk
Irani, L. (2015) · South Atlantic Quarterly · doi:10.1215/00382876-2831665 · Cited by 367
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000375277 · Cited by 360
Behind the Screen: Content Moderation in the Shadows of Social Media
ROBERTS, S. T. (2019) · Yale University Press · doi:10.2307/j.ctvhrcz0v · Cited by 339 · cited 1×
“Exposure to this content can be distressing (Roberts, 2019), although A. assures that she has found ways not to be personally affected.”
BackgroundThe Global Opportunity in Online Outsourcing
Kuek, S. C., Paradi-Guilford, C., Fayomi, T., Imaizumi, S., Ipeirotis, P. G., Pina, P., & Singh, M. (2015) · World Bank, Washington, DC eBooks · doi:10.1596/22284 · Cited by 306
Alpaydın, E. (2016) · Cited by 292
Heteromation, and Other Stories of Computing and Capitalism
Ekbia, H. R., & Nardi, B. A. (2017) · doi:10.7551/mitpress/10767.001.0001 · Cited by 285
Unpacking the Social Media Bot: A Typology to Guide Research and Policy
Gorwa, R., & Guilbeault, D. (2020) · Policy & Internet · doi:10.1002/poi3.184 · Cited by 270
The global gig economy: Towards a planetary labour market?
Graham, M. J., & Anwar, M. A. (2019) · First Monday · doi:10.5210/fm.v24i4.9913 · Cited by 215
Bechmann, A., & Bowker, G. C. (2019) · Big Data & Society · doi:10.1177/2053951718819569 · Cited by 162
Automation and Jobs: When Technology Boosts EmploymentPreprint
Bessen, J. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2935003 · Cited by 133
Micro-work, artificial intelligence and the automotive industry
Tubaro, P., & Casilli, A. A. (2019) · Journal of Industrial and Business Economics · doi:10.1007/s40812-019-00121-1 · Cited by 122
Self-citationKnowledge Map of the Virtual Economy
Lehdonvirta, V., & Ernkvist, M. (2011) · doi:10.1596/27361 · Cited by 114
Graham, M., Shaw, J., Berg, J., Colclough, C. J., Dewhurst, M., Gearhart, D., Jennings, P., McClenahan, G., Scholz, T. R., Silberman, M. ‘., et al. (2017) · doi:10.58704/5zv9-7s84 · Cited by 71
Regulation and the future of work: The employment relationship as an innovation facilitator
Aloisi, A., & STEFANO, V. D. (2020) · International Labour Review · doi:10.1111/ilr.12160 · Cited by 71
Boelaert, J., & Ollion, É. (2018) · Revue Française de Sociologie · doi:10.3917/e.rfs.593.0475 · Cited by 61
Boelaert, J., & Ollion, É. (2018) · Revue Française de Sociologie · doi:10.3917/rfs.593.0475 · Cited by 25
A Critical Examination of a Third Employment Category for On-Demand Work
Cherry, M. A., & Aloisi, A. (2018) · Cambridge University Press eBooks · doi:10.1017/9781108255882.024 · Cited by 12
Research works printed but not linked: 4
- Casilli A, 2019, En attendant les robots: Enquête sur le travail du clic
- Chui M, 2016, McKinsey Q, 2016, 1
- De Stefano V, 2016, Comparative Labor Law & Policy Journal, 37
- Silberman S, 2016, Comparative Labor Law & Policy Journal, 37
Cited by
Sorted by citations
The Ethical Implications of Artificial Intelligence (AI) For Meaningful Work
Bankins, S., & Formosa, P. (2023) · Journal of Business Ethics · doi:10.1007/s10551-023-05339-7 · Cited by 633
An interdisciplinary review of AI and HRM: Challenges and future directions
Pan, Y., & Froese, F. J. (2022) · Human Resource Management Review · doi:10.1016/j.hrmr.2022.100924 · Cited by 292
Gil de Zúñiga, H., Goyanes, M., & Durotoye, T. (2024) · Political Communication · doi:10.1080/10584609.2023.2290497 · Cited by 198
Artificial intelligence, human intelligence and hybrid intelligence based on mutual augmentation
Jarrahi, M. H., Lutz, C., & Newlands, G. (2022) · Big Data & Society · doi:10.1177/20539517221142824 · Cited by 156
Gmyrek, P., Berg, J., Bescond, D., & International Labour Organization. Research Department (2023) · doi:10.54394/fhem8239 · Cited by 140
Metadata
- Authors
- Tubaro, P., Casilli, A. A., & Coville, M.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Télécom Paris (FR)
- Université de Poitiers (FR)
- Corresponding author
- Tubaro, Paola
- Venue
- Big Data & Society
- Publisher
- SAGE Publications
- Published
- 2020-01
- Volume
- 7
- Issue
- 1
- Pages
- 205395172091977
- ISSN
- 2053-9517 (ISSN-L)
- OpenAlex
- W3018104257
- Cited by
- 262
- References
- 26
- Citations by year
- 2026: 44 · 2025: 61 · 2024: 48 · 2023: 38 · 2022: 45 · 2021: 19 · 2020: 7
- FWCI
- 36.46
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Big Data & Society (publishedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- HAL (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- France Stratégie
- MSH Paris Saclay (17-MA07)
- Force Ouvrière - Institut de recherches économiques et sociales
- Topics
- Social Sciences › Sociology and Political Science › Digital Economy and Work Transformation
- Social Sciences › Business, Management and Accounting › Industrial relations › Legal and Labor Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- AI development, data annotation, machine learning training data, precarious employment, AI verification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
