The Deskilling of Domain Expertise in AI Development
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Field workers, like farmers and radiologists, play a crucial role in dataset collection for AI models in low-resource settings. However, we know little about how field workers’ expertise is leveraged in dataset and model development. Based on 68 interviews with AI developers building for low-resource contexts, we find that developers reduced field workers to data collectors. Attributing poor data quality to worker practices, developers conceived of workers as corrupt, lazy, non-compliant, and as datasets themselves, pursuing surveillance and gamification to discipline workers to collect better quality data. Even though models sought to emulate the expertise of field workers, AI developers treated workers as non-essential and deskilled their expertise in service of building machine intelligence. We make the case for why field workers should be recognised as domain experts and re-imagine domain expertise as an essential partnership for AI development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Algorithms at Work: The New Contested Terrain of Control
Kellogg, K. C., Valentine, M. A., & Christin, A. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0174 · Cited by 2,732
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Suchman, L. (1995) · Communications of the ACM · doi:10.1145/223248.223263 · Cited by 991 · cited 1×
“More generally, it is indeed a well-recognised problem that computer scientists and engineers often do not recognise the expertise of workers, despite their crucial contributions [34, 79].”
BackgroundTurkopticon: interrupting worker invisibility in amazon mechanical turk
Irani, L. C., & Silberman, M. S. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470742 · Cited by 798
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781 · cited 4×
“lack of agency of field workers [41], done with inadequate training [12, 55, 58, 63], or mismatched incentives [12, 72, 87].”
BackgroundMethodInfluentialSelf-citationThe cultural work of microwork
Irani, L. (2015) · New Media & Society · doi:10.1177/1461444813511926 · Cited by 568 · cited 1×
“The workers of ML and AI are divided between the ”coding elite” [10] who do the innovative work and the “cybertariat” [36], who does the “menial work” of building data that feed or train the algorithms [38].”
BackgroundData Augmentation for Low-Resource Neural Machine Translation
Fadaee, M., Bisazza, A., & Monz, C. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p17-2090 · Cited by 266 · cited 1×
“An active area of research is focused on coping with limited datasets through technical solutions, including dimensionality reduction [95] and data augmentation [22].”
BackgroundSharing Knowledge and Expertise: The CSCW View of Knowledge Management
Ackerman, M. S., Dachtera, J., Pipek, V., & Wulf, V. (2013) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-013-9192-8 · Cited by 229
Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions
Alkhatib, A., & Bernstein, M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300760 · Cited by 188
Bringing the People Back In: Contesting Benchmark Machine Learning Datasets
Denton, R., Hanna, A., Amironesei, R., Smart, A., Nicole, H., & Scheuerman, M. K. (2020) · arXiv · doi:10.48550/arxiv.2007.07399 · Cited by 100
Cited by
Ross, S. I., Martinez, F., Houde, S., Muller, M., & Weisz, J. D. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584037 · Cited by 258
Natali, C., Marconi, L., Dias Duran, L. D., & Cabitza, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11352-1 · Cited by 143
“In a broad perspective, Sambasivan and Veeraraghavan ( 2022 ) have highlighted how AI development, particularly in low-resource contexts, often results in the deskilling of local domain experts: in these scenarios, domain experts are frequently relegated to the role of data collectors, performing…”
Varanasi, R. A., & Goyal, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580903 · Cited by 52
A hunt for the Snark: Annotator Diversity in Data Practices
Kapania, S., Taylor, A. S., & Wang, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580645 · Cited by 50
“Sambasivan and Veeraraghavan [81] demonstrated that even in settings where data production is reliant on expert fieldworkers ( e.g., farmers and radiologists) and their highly situated knowledge, practitioners still reduced field workers to data collectors, and attributed poor data quality to their…”
BackgroundFrom Bias to Repair: Error as a Site of Collaboration and Negotiation in Applied Data Science Work
Lin, C. K., & Jackson, S. J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579607 · Cited by 22
“Nithya Sambasivan and Rajesh Veeraraghavan have shown how AI developers may moralize error (and disclaim responsibility), characterizing field workers collecting training data as lazy, corrupt, and careless, further justifying the use of surveillance, automated checking tools, and cross-verification to eliminate error…”
Background
Metadata
- Authors
- Sambasivan, N., & Veeraraghavan, R.
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-14
- OpenAlex
- W4225001019
- Cited by
- 78
- References
- 14
- FWCI
- 14.64
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Deskilling of Domain Expertise in AI Development. https://app.cochat.ai/papers/10.1145/3491102.3517578
- The paper
- Sambasivan, N., & Veeraraghavan, R. (2022). The Deskilling of Domain Expertise in AI Development. CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517578
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.