CoChat
Sign Up
Cited by 6Ethics and Social Impacts of AIDigital Economy and Work TransformationLegal and Labor Studies

Match Made by Humans: A Critical Enquiry into Human-Machine Configurations in Data Labelling

Open access
Chandhiramowuli, Srravya, Chaudhuri, Bidisha·2023·Proceedings of the Annual Hawaii International Conference on System Sciences
DOI 10.24251/hicss.2023.251
Open this paper in CoChat — free accountOpen PDF

Abstract

We present a critical ethnographic study of data labelling conducted in a Bangalore-based AI start-up. Labelled datasets are primarily produced by human data workers. We explore how humans and machines are configured together in data labelling and what are the demands placed on human workers, including on their body and cognition, while being assigned in the service of machine intelligence. We also show how these human-machine configurations sustain and reproduce the seamless functioning of apparently “autonomous” AI as a normative vision. Though labelled datasets are an indispensable prerequisite to creating ML/AI-based systems, the human labour that produces these datasets cannot be acknowledged fully if the techno-entrepreneurial vision of “self-learning” machine intelligence is to be celebrated and sustained. In pursuit of a normative position of what AI should be, we are left with a denial of how AI is actually produced now.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.24251/hicss.2023.251

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Technology companies often outsource data-related tasks through BPO firms or platforms that operate across national borders.

    “When technology companies require data-related tasks to support AI development, they often outsource this work through Business Process Outsourcing (BPO) firms [138] or platforms [4, 67] that typically operate across national borders [23].”

    From 'The plan is just survival': Data Work in Kenya and the Regime of Entrapment (2026) · cites this paper as [23]

    Section 2 (Relevant Work; page unavailable) says AI data work is outsourced to the Global South and that Global North AI/ML companies employ data workers in lower-income countries. This supports cross-border outsourcing, but the paper does not identify BPO firms as the intermediary—the part of the citing sentence to which this source is attached. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 22

  1. Ajunwa, I., & Greene, D. (2019). Platforms at work: Automated hiring platforms and other new intermediaries in the organization of work. In Work and labor in the digital age. Emerald Publishing Limited.
  2. Autor, D. (2015). Why are there still so many jobs? The history and future of workplace automation. Journal of Economic Perspectives, 29(3), 3-30.
  3. Bilić, P. (2016). Search algorithms, hidden labour and information control. Big Data & Society, 3(1), 2053951716652159. https://doi.org/10.1177/2053951716652159 · DOI
  4. Ekbia, H. R., & Nardi, B. A. (2017). Heteromation, and other stories of computing and capitalism. MIT Press.
  5. Graham, M., Hjorth, I., & Lehdonvirta, V. (2017). Digital labour and development: Impacts of global digital labour platforms and the gig economy on worker livelihoods. Transfer: European Review of Labour and Research, 23(2), 135-162. https://doi.org/10.1177/1024258916687250 · DOI · Paper page
  6. Gupta, H. (2019). Testing the future: Gender and technocapitalism in start-up India. Feminist Review, 123(1), 74-88.
  7. Irani, L. (2015a). The cultural work of microwork. New Media & Society, 17(5), 720-739.
  8. Irani, L. (2015c). Difference and dependence among digital workers: The case of Amazon Mechanical Turk. South Atlantic Quarterly, 114(1), 225-234.
  9. Jasanoff, S. (2017). Virtual, visible, and actionable: Data assemblages and the sightlines of justice. Big Data & Society, 4(2), 2053951717724477.
  10. Lee, M. K., Kusbit, D., Metsky, E., & Dabbish, L. (2015). Working with machines: The impact of algorithmic and data-driven management on human workers. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 1603-1612.
  11. Miceli, M., Schuessler, M., & Yang, T. (2020). Between subjectivity and imposition: Power dynamics in data annotation for computer vision. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW2), 1-25.
  12. Miceli, M., Posada, J., & Yang, T. (2021). Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power? ArXiv Preprint ArXiv:2109.08131. · arXiv
  13. Miceli, M., & Posada, J. (2022). The Data-Production Dispositif. arXiv preprint arXiv:2205.11963. · arXiv
  14. Moore, P., & Robinson, A. (2016). The quantified self: What counts in the neoliberal workplace. New Media & Society, 18(11), 2774-2792.
  15. Pietrobelli, C., & Rabellotti, R. (2011). Global value chains meet innovation systems: Are there learning opportunities for developing countries? World Development, 39(7), 1261-1269.
  16. Posada, J. (2020). The Future of Work Is Here: Toward a Comprehensive Approach to Artificial Intelligence and Labour. ArXiv Preprint ArXiv:2007.05843. · arXiv
  17. Poster, W. R., Crain, M., & Cherry, M. A. (2016). 1. Introduction: Conceptualizing Invisible Labor. In Invisible Labor (pp. 3-27). University of California Press.
  18. Raval, N. (2021). Interrupting invisibility in a global world | ACM Interactions. Interactions, 28(4), 27-31.
  19. Roberts, S. T. (2016). Commercial content moderation: Digital laborers' dirty work. In Noble, S.U. and Tynes, B. (Eds.), The intersectional internet: Race, sex, class and culture online (pp. 147-159). New York: Peter Lang.
  20. Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021). "Everyone wants to do the model work, not the data work": Data Cascades in High-Stakes AI. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1-15.
  21. Ustek-Spilda, F., Powell, A., & Nemorin, S. (2019). Engaging with ethics in Internet of Things: Imaginaries in the social milieu of technology developers. Big Data & Society, 6(2), 2053951719879468. https://doi.org/10.1177/2053951719879468 · DOI
  22. Wood, A. J., Lehdonvirta, V., & Graham, M. (2018). Workers of the Internet unite? Online freelancer organisation among remote gig economy workers in six Asian and African countries. New Technology, Work and Employment, 33(2), 95-112.

Cited by

Sorted by citations

Authors
Chandhiramowuli, S., & Chaudhuri, B.
Affiliations
International Institute of Information Technology Bangalore (IN)
ORCID
Venue
Proceedings of the Annual Hawaii International Conference on System Sciences
Publisher
Hawaii International Conference on System Sciences
Published
2023
ISSN
  • 1530-1605 (ISSN-L)
  • 2572-6862
DOI
10.24251/hicss.2023.251
OpenAlex
W4391904232
Cited by
6
References
5
Citations by year
2026: 3 · 2025: 2 · 2024: 1
FWCI
1.50
Type
conference paper
Language
en
Open access
Licence
public-domain
Locations
  • Hawaii International Conference on System Sciences (HICSS) (publishedVersion, public-domain)
  • Journal of the Association for Information Systems (submittedVersion)
  • ScholarSpace (University of Hawaii at Manoa) (submittedVersion, cc-by-nc-nd)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Sociology and Political Science › Digital Economy and Work Transformation
  • Social Sciences › Business, Management and Accounting › Industrial relations › Legal and Labor Studies
Keywords
data annotation, AI startups, annotated dataset, machine learning, AI autonomy, human labor, body and cognition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review