CoChat
Sign Up
PreprintCited by 1Digital Humanities and ScholarshipCulinary Culture and TourismResearch Data Management Practices

The Human Labour of Data Work: Capturing Cultural Diversity through World Wide Dishes

Open access
Hall, Siobhan Mackenzie, Dalal, Samantha, Sefala, Raesetje, Yuehgoh, Foutse, Alaagib, Aisha, Hamzaoui, Imane, Ishida, Shu, Magomere, Jabez +3 more·2025·arXiv
DOI 10.48550/arxiv.2502.05961arXiv 2502.05961
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper provides guidance for building and maintaining infrastructure for participatory AI efforts by sharing reflections on building World Wide Dishes (WWD), a bottom-up, community-led image and text dataset of culinary dishes and associated cultural customs. We present WWD as an example of participatory dataset creation, where community members both guide the design of the research process and contribute to the crowdsourced dataset. This approach incorporates localised expertise and knowledge to address the limitations of web-scraped Internet datasets acknowledged in the Participatory AI discourse. We show that our approach can result in curated, high-quality data that supports decentralised contributions from communities that do not typically contribute to datasets due to a variety of systemic factors. Our project demonstrates the importance of participatory mediators in supporting community engagement by identifying the kinds of labour they performed to make WWD possible. We surface three dimensions of labour performed by participatory mediators that are crucial for participatory dataset construction: building trust with community members, making participation accessible, and contextualising community values to support meaningful data collection. Drawing on our findings, we put forth five lessons for building infrastructure to support future participatory AI efforts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2502.05961

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Rather than treating PwD as a singular or universal identity, the authors acknowledge the distinct social identities and intersectionalities within each community.

    “Rather than treating PwD as a singular or universal identity -often reinforced by homogenizing labels such as 'dwarfism -we acknowledge the distinct social identities and intersectionalities within each community (cf. [53, 90]).”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [53]

    Section 6.3 says distinct cultures should not be lumped into a “single flattened snapshot” and describes an underrepresented ethnic community within Nigeria. That supports the broader point about avoiding homogenizing labels, but this paper addresses cultural representation, not PwD identities or their intersectionalities. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Researchers have previously problematized anonymous crowdsourced data annotation approaches as being unspecific about who's worldviews and beliefs become manifest in the data.

    “Researchers have previously problematized anonymous crowdsourced data annotation approaches [31, 53, 65] as being unspecific about who's worldviews and beliefs become manifest in the data [11, 17, 51, 78, 90, 91].”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [53]

    Section 2.1 says datasets carry their creators’ worldviews and beliefs, while Section 2.2 criticizes top-down crowdsourcing for excluding some perspectives. The paper does not specifically argue that anonymous annotation leaves it unclear whose worldviews and beliefs appear in the data; Section 4.3.2 describes accepting anonymous submissions while still collecting demographic information. No page numbers were provided in our copy. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Approaches that enable communities to define and curate datasets remain underexplored.

    “Despite calls for more inclusive data-centric AI (e.g., [11, 31, 53, 96, 107]), approaches that enable communities to define and curate datasets remain underexplored.”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [53]

  • Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups, including people with disabilities.

    “Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups [15, 21, 36, 41, 42, 45, 53, 66, 72, 91], including PwD [10, 12, 39, 48, 54, 62, 77, 113].”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [53]

  • Data is not only relational, but also shaped by its creation context, whereby datasets reflect the worldviews and beliefs of their creators who decide what deserves capturing and how information is classified.

    “This requires recognizing that data is not only relational, but also shaped by its creation context, whereby datasets reflect the worldviews and beliefs of their creators who decide what deserves capturing and how information is classified [17, 53].”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [53]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 108

  1. n. d.]. AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languages, author = Ogundepo, Odunayo and Gwadabe, Tajuddeen R and Rivera, Clara E and Clark, Jonathan H and Ruder, Sebastian and Adelani, David Ifeoluwa and Dossou, Bonaventure FP and Diop, Abdou Aziz and Sikasote, Claytone and Hacheme, Gilles and others, year = 2023, journal = arXiv preprint arXiv:2305.06897. ([n. d.]). · arXiv
  2. Blodgett, Su Lin and Lopez, Gilsinia and Olteanu, Alexandra and Sim, Robert and Wallach, Hanna, year = 2021, booktitle = Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages = 1004-1015.
  3. David Ifeoluwa Adelani, Graham Neubig, Sebastian Ruder, Shruti Rijhwani, Michael Beukman, Chester Palen-Michel, Constantine Lignos, Jesujoba O Alabi, Shamsuddeen H Muhammad, Peter Nabende, et al. 2022. MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition. arXiv preprint arXiv:2210.12391 (2022). · arXiv
  4. Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Singh, Ashutosh Dwivedi, Al- ham Fikri Aji, Jacki O'Neill, Ashutosh Modi, and Monojit Choudhury. 2024. Towards Measuring and Modeling "Culture" in LLMs: A Survey. arXiv preprint arXiv:2403.15412 (2024). · arXiv
  5. Peter M Asaro. 2000. Transforming society by transforming technology: the science and politics of participatory design. Accounting, Management and Information Technologies 10, 4 (2000), 257-290.
  6. Liam Bannon, Jeffrey Bardzell, and Susanne Bødker. 2018. Reimagining Participatory Design. Interactions 26, 1 (2018), 26-32.
  7. Chelsea Barabas, Colin Doyle, JB Rubinovitz, and Karthik Dinakar. 2020. Studying Up: Reorienting the study of algorithmic fairness around issues of power. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* '20). Association for Computing Machinery, New York, NY, USA, 167-176. doi:10.1145/3351095.3372859 · DOI
  8. Ruha Benjamin. 2016. Informed Refusal: Toward a Justicebased Bioethics. Science, Technology, & Human Values 41, 6 (2016), 967-990.
  9. R Benjamin. 2019. Race after Technology: Abolitionist Tools for the New Jim Code. Cambridge and Medford: Polity Press.
  10. Sebastian Benthall and Bruce D Haynes. 2019. Racial categories in machine learning. In Proceedings of the conference on Fairness, Accountability, and Transparency. 289-298.
  11. Hugo Berg, Siobhan Mackenzie Hall, Yash Bhalgat, Wonsuk Yang, Hannah Rose Kirk, Aleksandar Shtedritski, and Max Bain. 2022. A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning. arXiv preprint arXiv:2203.11933 (2022). · arXiv
  12. Mukul Bhutani, Kevin Robinson, Vinodkumar Prabhakaran, Shachi Dave, and Sunipa Dev. 2024. SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypes. arXiv preprint arXiv:2403.05696 (2024). · arXiv
  13. Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Díaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed. 2022. Power to the People? Opportunities and Challenges for Participatory AI. arXiv:2209.07572 [cs] doi:10.1145/3551624.3555290 · DOI · Paper page
  14. Abeba Birhane and Vinay Uday Prabhu. 2021. Large image datasets: A pyrrhic win for computer vision?. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1536-1546.
  15. Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. 2021. Multimodal datasets: misogyny, pornography, and malignant stereotypes. arXiv preprint arXiv:2110.01963 (2021). · arXiv
  16. Janet Blake. 2000. On Defining the Cultural Heritage. International & Comparative Law Quarterly 49, 1 (2000), 61-85.
  17. Susanne Bødker, Christian Dindler, Ole S Iversen, and Rachel C Smith. 2022. What Can We Learn from the History of Participatory Design? In Participatory Design. Springer, 15-29.
  18. Marcel Borowski, Bjarke V Fog, Carla F Griggio, James R Eagan, and Clemens N Klokmose. 2023. Between Principle and Pragmatism: Reflections on Prototyping Computational Media with Webstrates. ACM Transactions on Computer- Human Interaction 30, 4 (2023), 1-53.
  19. Geoffrey C Bowker. 2000. Sorting things out: Classification and its consequences. MIT press.
  20. Tone Bratteteig and Ina Wagner. 2016. Unpacking the Notion of Participation in Participatory Design. Computer Supported Cooperative Work (CSCW) 25 (2016), 425-475.
  21. Jed R Brubaker and Gillian R Hayes. 2011. SELECT* FROM USER: infrastructure and socio-technical representation. In Proceedings of the ACM 2011 conference on Computer supported cooperative work. 369-378.
  22. Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research, Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, 77-91. https://proceedings.mlr.press/ v81/buolamwini18a.html
  23. Heewon Chang. 2021. Individual and Collaborative Autoethnography for Social Science Research. In Handbook of autoethnography. Routledge, 53-65.
  24. John Cheney-Lippold. 2017. We Are Data: Algorithms and the Making of our Digital Selves. In We Are Data. New York University Press.
  25. Eric Corbett, Emily Denton, and Sheena Erete. 2023. Power and Public Participation in AI. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1-13.
  26. Geoffrey Currie, Johnathan Hewis, Elizabeth Hawk, and Eric Rohren. 2024. Gender and Ethnicity Bias of Text-to-Image Generative Artificial Intelligence in Medical Imaging, Part 1: Preliminary Evaluation. Journal of Nuclear Medicine Technology (2024).
  27. Samantha Dalal, Siobhan Mackenzie Hall, and Nari Johnson. 2024. Provocation: Who benefits from" inclusion" in Generative AI? arXiv preprint arXiv:2411.09102 (2024). · arXiv
  28. Dipto Das, Carsten Østerlund, and Bryan Semaan. 2021. "Jol" or "Pani"?: How Does Governance Shape a Platform's Identity? Proc. ACM Hum.-Comput. Interact. 5, CSCW2, Article 473 (Oct. 2021), 25 pages. doi:10.1145/3479860 · DOI
  29. Dipto Das and Bryan Semaan. 2022. Collaborative Identity Decolonization as Reclaiming Narrative Agency: Identity Work of Bengali Communities on Quora. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI '22). Association for Computing Machinery, New York, NY, USA, Article 236, 23 pages. doi:10.1145/3491102.3517600 · DOI
  30. Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023. The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1-23.
  31. Remi Denton, Mark Díaz, Ian Kivlichan, Vinodkumar Prabhakaran, and Rachel Rosen. 2021. Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation. arXiv preprint arXiv:2112.04554 (2021). · arXiv
  32. Audrey Desjardins, Oscar Tomico, Andrés Lucero, Marta E Cecchinato, and Carman Neustaedter. 2021. Introduction to the special issue on first-person methods in HCI. ACM Transactions on Computer-Human Interaction (TOCHI) 28, 6, 1-12.
  33. Catherine D'Ignazio. 2024. Counting feminicide: Data feminism in action. MIT Press.
  34. Catherine D'ignazio and Lauren F Klein. 2023. Data feminism. MIT press.
  35. Emile Durkheim and Marcel Mauss. 2009. Primitive classification (Routledge revivals). Routledge.
  36. Steven Epstein. 2008. The Rise of Recruitmentology' Clinical Research, Racial Knowledge, and the Politics of Inclusion and Difference. Social Studies of Science 38, 5 (2008), 801-832.
  37. Sheena Erete, Yolanda A. Rankin, and Jakita O. Thomas. 2021. I Can't Breathe: Reflections from Black Women in CSCW and HCI. Proc. ACM Hum.-Comput. Interact. 4, CSCW3, Article 234 (Jan. 2021), 23 pages. doi:10.1145/3432933 · DOI
  38. Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, and Vaishaal Shankar. 2023. Data Filtering Networks. arXiv preprint arXiv:2309.17425 (2023). · arXiv
  39. ∀, Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila, Timi Fasubaa, Tajudeen Kolawole, Taiwo Fagbo- hungbe, Solomon Oluwole Akinola, Shamsuddeen Hassan Muhammad, Salomon Kabongo, Salomey Osei, et al. 2020. Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages. arXiv preprint arXiv:2010.02353 (2020). · arXiv
  40. ∀, Iroro Orife, Julia Kreutzer, Blessing Sibanda, Daniel Whitenack, Kathleen Siminyu, Laura Martinus, Jamiil Toure Ali, Jade Abbott, Vukosi Marivate, Salomon Kabongo, et al. 2020. Masakhane-Machine Translation For Africa. arXiv preprint arXiv:2003.11529 (2020). · arXiv
  41. Andrew L Friedman and Dominic S Cornford. 1989. Computer Systems Development: History Organization and Implementation. John Wiley & Sons, Inc.
  42. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. 2021. Datasheets for Datasets. Commun. ACM 64, 12 (2021), 86-92.
  43. Clifford Geertz. 2008. Thick Description: Toward an Interpretive Theory of Culture. In The cultural geography reader. Routledge, 41-51.
  44. R Stuart Geiger, Kevin Yu, Yanlai Yang, Mindy Dai, Jie Qiu, Rebekah Tang, and Jenny Huang. 2020. Garbage In, Garbage Out? Do Machine Learning Application Papers in Social Computing Report Where Human-Labeled Training Data Comes From?. In Proceedings of the 2020 conference on Fairness, Accountability, and Transparency. 325-336.
  45. Tarleton Gillespie. 2014. The Relevance of Algorithms. Media technologies: Essays on communication, materiality, and society 167, 2014 (2014), 167.
  46. Tarleton Gillespie. 2024. Generative AI and the politics of visibility. Big Data & Society 11, 2 (2024), 20539517241252131.
  47. Jonathan Grudin. 1988. Why CSCW Applications Fail: Problems in the Design and Evaluation of Organizational Interfaces. In Proceedings of the 1988 ACM conference on Computer-supported cooperative work. 85-93.
  48. Akhil Gupta and James Ferguson. 2008. Beyond "Culture": Space, Identity, and the Politics of Difference. In The cultural geography reader. Routledge, 72-79.
  49. Siobhan Mackenzie Hall, Fernanda Gonçalves Abrantes, Hanwen Zhu, Grace Sodunke, Aleksandar Shtedritski, and Hannah Rose Kirk. 2024. VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution. Advances in Neural Information Processing Systems 36 (2024).
  50. Donna Haraway. 2013. Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective 1. In Women, science, and technology. Routledge, 455-472.
  51. Christina Harrington, Sheena Erete, and Anne Marie Piper. 2019. Deconstructing Community-Based Collaborative Design: Towards More Equitable Participatory Design Engagements. Proceedings of the ACM on human-computer interaction 3, CSCW (2019), 1-25.
  52. Tajanae Harris. 2024. Data for Whom, Data from Whom: How Social Movements Might Create Value for Their Community Data Practices. XRDS 30, 4 (June 2024), 31-35. doi:10.1145/3665597 · DOI
  53. Sarah Homewood. 2023. Self-Tracking to Do Less: An Autoethnography of Long COVID That Informs the Design of Pacing Technologies. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-14.
  54. Rachel Hong, William Agnew, Tadayoshi Kohno, and Jamie Morgenstern. 2024. Who's in and who's out? A case study of multimodal CLIP-filtering in DataComp. In Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1-17.
  55. Rachel Hong, Tadayoshi Kohno, and Jamie Morgenstern. 2023. Evaluation of targeted dataset collection on racial equity in face recognition. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (Montréal, QC, Canada) (AIES '23). Association for Computing Machinery, New York, NY, USA, 531-541. doi:10.1145/3600211.3604662 · DOI
  56. Noura Howell, Audrey Desjardins, and Sarah Fox. 2021. Cracks in the success narrative: Rethinking failure in design research through a retrospective trioethnography. ACM Transactions on Computer-Human Interaction (TOCHI) 28, 6 (2021), 1-31.
  57. Lilly C Irani and M Six Silberman. 2016. Stories We Tell About Labor: Turkopticon and the Trouble with" Design". In Proceedings of the 2016 CHI conference on human factors in computing systems. 4573-4586.
  58. Azra Ismail and Neha Kumar. 2018. Engaging Solidarity in Data Collection Practices for Community Health. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (Nov. 2018), 1-24. doi:10.1145/3274345 · DOI · Paper page
  59. Akshita Jha, Vinodkumar Prabhakaran, Remi Denton, Sarah Laszlo, Shachi Dave, Rida Qadri, Chandan Reddy, and Sunipa Dev. 2024. ViSAGe: A Global-Scale Analysis of Visual Stereotypes in Text-to-Image Generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 12333-12347.
  60. Shivani Kapania, Alex S Taylor, and Ding Wang. 2023. A Hunt for the Snark: Annotator Diversity in Data Practices. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg Germany, 1-15. doi:10.1145/3544548.3580645 · DOI · Paper page
  61. Jared Katzman, Angelina Wang, Morgan Scheuerman, Su Lin Blodgett, Kristen Laird, Hanna Wallach, and Solon Barocas. 2023. Taxonomizing and Measuring Representational Harms: A Look at Image Tagging. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 14277-14285.
  62. Hannah Rose Kirk, Alexander Whitefield, Paul Röttger, Andrew Bean, Katerina Margatina, Juan Ciro, Rafael Mosquera, Max Bartolo, Adina Williams, He He, et al. 2024. The PRISM Alignment Project: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models. arXiv preprint arXiv:2404.16019 (2024). · arXiv
  63. Aniket Kittur, Jeffrey V. Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease, and John Horton. 2013. The Future of Crowd Work. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work (San Antonio, Texas, USA) (CSCW '13). Association for Computing Machinery, New York, NY, USA, 1301-1318. doi:10.1145/2441776.2441923 · DOI · Paper page
  64. Christopher A Le Dantec and Sarah Fox. 2015. Strangers at the Gate: Gaining Access, Building Rapport, and Co- Constructing Community-Based Research. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing. 1348-1358.
  65. Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, et al. 2024. A large-scale audit of dataset licensing and attribution in AI. Nature Machine Intelligence 6, 8 (2024), 975-987.
  66. Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. 2024. Stable Bias: Evaluating Societal Representations in Diffusion Models. Advances in Neural Information Processing Systems 36 (2024).
  67. Kelly Mack, Maitraye Das, Dhruv Jain, Danielle Bragg, John Tang, Andrew Begel, Erin Beneteau, Josh Urban Davis, Abraham Glasser, Joon Sung Park, and Venkatesh Potluri. 2021. Mixed Abilities and Varied Experiences: a group autoethnography of a virtual summer internship. In Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility (Virtual Event, USA) (ASSETS '21). Association for Computing Machinery, New York, NY, USA, Article 21, 13 pages. doi:10.1145/3441852.3471199 · DOI · Paper page
  68. Jabez Magomere, Shu Ishida, Tejumade Afonja, Aya Salama, Daniel Kochin, Yuehgoh Foutse, Imane Hamzaoui, Raesetje Sefala, Aisha Alaagib, Samantha Dalal, et al. 2025. The World Wide recipe: A community-centred framework for fine-grained data collection and regional bias operationalisation. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency. 246-282.
  69. Daniela Massiceti, Luisa Zintgraf, John Bronskill, Lida Theodorou, Matthew Tobias Harris, Edward Cutrell, Cecily Morrison, Katja Hofmann, and Simone Stumpf. 2021. ORBIT: A real-world few-shot dataset for teachable object recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 10818-10828.
  70. David W McMillan and David M Chavis. 1986. Sense of Community: A Definition and Theory. Journal of community psychology 14, 1 (1986), 6-23.
  71. Milagros Miceli and Julian Posada. 2022. The Data-Production Dispositif. Proc. ACM Hum.-Comput. Interact. 6, CSCW2, Article 460 (Nov. 2022), 37 pages. doi:10.1145/3555561 · DOI
  72. Milagros Miceli, Julian Posada, and Tianling Yang. 2022. Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power? Proc. ACM Hum.-Comput. Interact. 6, GROUP, Article 34 (Jan. 2022), 14 pages. doi:10.1145/ 3492853 · DOI · Paper page
  73. Naja Holten Møller, Claus Bossen, Kathleen H. Pine, Trine Rask Nielsen, and Gina Neff. 2020. Who Does the Work of Data? Interactions 27, 3 (April 2020), 52-55. doi:10.1145/3386389 · DOI
  74. Esther Mwema and Abeba Birhane. 2025. Undersea cables in Africa: The new frontiers of digital colonialism. First Monday 29, 4 (2025). doi:10.5210/fm.v29i4.13637 [Accessed 13 January 2025]. · DOI
  75. Safiya Umoja Noble. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. New York University Press, New York, USA. doi:doi:10.18574/nyu/9781479833641.001.0001 · DOI
  76. Victor Ojewale, Ryan Steed, Briana Vecchione, Abeba Birhane, and Inioluwa Deborah Raji. 2024. Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling. arXiv:2402.17861 (March 2024). arXiv:2402.17861 [cs] · arXiv
  77. Kazuo Okamura, Masayo Fujimoto, Wanda J. Orlikowski, and Joanne Yates. 1995. Helping CSCW Applications Succeed: The Role of Mediators in the Context of Use. The Information Society 11, 3 (July 1995), 157-172. doi:10.1080/ 01972243.1995.9960190 · DOI
  78. Jennifer Pierre, Roderic Crooks, Morgan Currie, Britt Paris, and Irene Pasquetto. 2021. Getting Ourselves Together: Data-centered participatory design research & epistemic burden. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-11.
  79. Trevor J Pinch and Wiebe E Bijker. 1984. The Social Construction of Facts and Artefacts: Or How the Sociology of Science and the Sociology of Technology Might Benefit Each Other. Social studies of science 14, 3 (1984), 399-441.
  80. Julian Alberto Posada Gutierrez. 2022. The Coloniality of Data Work: Power and Inequality in Outsourced Data Production for Machine Learning. Ph. D. Dissertation. University of Toronto.
  81. Vinodkumar Prabhakaran, Rida Qadri, and Ben Hutchinson. 2022. Cultural Incongruencies in Artificial Intelligence. arXiv preprint arXiv:2211.13069 (2022). · arXiv
  82. Satish K. Puri and Sundeep Sahay. 2007. Role of ICTs in participatory development: An Indian experience. Information Technology for Development 13, 2 (2007), 133-160. doi:10.1002/itdj.20058 · DOI
  83. Rida Qadri, Mark Diaz, Ding Wang, and Michael Madaio. 2025. The Case for 'Thick Evaluations' of Cultural Representation in AI. arXiv preprint arXiv:2503.19075 (2025). · arXiv
  84. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning Transferable Visual Models From Natural Language Supervision. In International conference on machine learning. PMLR, 8748-8763.
  85. Vikram V. Ramaswamy, Sing Yu Lin, Dora Zhao, Aaron B. Adcock, Laurens van der Maaten, Deepti Ghadi- yaram, and Olga Russakovsky. 2023. GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognition. arXiv:2301.02560 [cs.CV] https://arxiv.org/abs/2301.02560 · arXiv
  86. Peter Reason and Hilary Bradbury. 2008. The SAGE Handbook of Action Research. SAGE Publications Ltd, 1 Oliver's Yard, 55 City Road, London England EC1Y 1SP United Kingdom. doi:10.4135/9781848607934 · DOI
  87. William A Gaviria Rojas, Sudnya Diamos, Keertan Ranjan Kini, David Kanter, Vijay Janapa Reddi, and Cody Coleman. 2022. The Dollar Street Dataset: Images Representing the Geographic and Socioeconomic Diversity of the World. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track. https://openreview. net/forum?id=qnfYsave0U4
  88. David Romero, Chenyang Lyu, Haryo Akbarianto Wibowo, Teresa Lynn, Injy Hamed, Aditya Nanda Kishore, Aishik Mandal, Alina Dragonetti, Artem Abzaliev, Atnafu Lambebo Tonja, et al. 2024. CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark. arXiv preprint arXiv:2406.05967 (2024). · arXiv
  89. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender Bias in Coreference Resolution. arXiv preprint arXiv:1804.09301 (2018). · arXiv
  90. Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo. 2021. "Everyone wants to do the model work, not the data work": Data Cascades in High-Stakes AI. In proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-15.
  91. Morgan Klaus Scheuerman and Jed R. Brubaker. 2024. Products of Positionality: How Tech Workers Shape Identity Concepts in Computer Vision. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, Honolulu HI USA, 1-18. doi:10.1145/3613904.3641890 · DOI
  92. Morgan Klaus Scheuerman, Alex Hanna, and Emily Denton. 2021. Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (Oct. 2021), 1-37. doi:10.1145/3476058 · DOI · Paper page
  93. Morgan Klaus Scheuerman, Jacob M Paul, and Jed R. Brubaker. 2019. How Computers See Gender: An Evaluation of Gender Classification in Commercial Facial Analysis and Image Labeling Services. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 144 (2019), 33 pages. doi:10.1145/3359246 · DOI · Paper page
  94. Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. 2022. LAION-5B: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems 35 (2022), 25278-25294.
  95. Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D Sculley. 2017. No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World. arXiv preprint arXiv:1711.08536 (2017). · arXiv
  96. Shivalika Singh, Freddie Vargus, Daniel D'souza, Börje F Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura O'Mahony, et al. 2024. Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning. arXiv preprint arXiv:2402.06619 (2024). · arXiv
  97. Susan Leigh Star. 1990. Power, technology and the phenomenology of conventions: on being allergic to onions. The Sociological Review 38, 1_suppl (1990), 26-56.
  98. Susan Leigh Star and Anselm Strauss. 1999. Layers of Silence, Arenas of Voice: The Ecology of Visible and Invisible Work. Computer supported cooperative work (CSCW) 8 (1999), 9-30.
  99. Cella M Sum, Anh-Ton Tran, Jessica Lin, Rachel Kuo, Cynthia L Bennett, Christina Harrington, and Sarah E Fox. 2023. Translation as (Re) mediation: How Ethnic Community-Based Organizations Negotiate Legitimacy. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-14.
  100. Harini Suresh, Emily Tseng, Meg Young, Mary Gray, Emma Pierson, and Karen Levy. 2024. Participation in the Age of Foundation Models. In The 2024 ACM Conference on Fairness, Accountability, and Transparency. 1609-1621.
  101. Latanya Sweeney. 2013. Discrimination in Online Ad Delivery. Commun. ACM 56, 5 (2013), 44-54.
  102. Atnafu Lambebo Tonja, Bonaventure FP Dossou, Jessica Ojo, Jenalea Rajab, Fadel Thior, Eric Peter Wairagala, Aremu Anuoluwapo, Pelonomi Moiloa, Jade Abbott, Vukosi Marivate, et al. 2024. InkubaLM: A small language model for low-resource African languages. arXiv preprint arXiv:2408.17024 (2024). · arXiv
  103. UNESCO. 2003. Convention for the Safeguarding of the Intangible Cultural Heritage. https://ich.unesco.org/ en/convention Adopted by the General Conference of the United Nations Educational, Scientific and Cultural Organization, 32nd Session, Paris, 17 October 2003.
  104. Elizabeth Anne Watkins. 2023. Face Work: A Human-Centered Investigation into Facial Verification in Gig Work. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1-24.
  105. Laura Weidinger, Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Juan Mateos-Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach, et al. 2023. Sociotechnical Safety Evaluation of Generative AI Systems. arXiv preprint arXiv:2310.11986 (2023). · arXiv
  106. Genta Indra Winata, Frederikus Hudi, Patrick Amadeus Irawan, David Anugraha, Rifki Afina Putri, Yutong Wang, Adam Nohejl, Ubaidillah Ariq Prathama, Nedjma Ousidhoum, Afifa Amriani, et al. 2024. WorldCuisines: A Massive- Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines. arXiv preprint arXiv:2410.12705 (2024). · arXiv
  107. Jordan Wirfs-Brock, Alli Fam, Laura Devendorf, and Brian Keegan. 2021. Examining Narrative Sonification: Using First-Person Retrospection Methods to Translate Radio Production to Interaction Design. ACM Transactions on Computer-Human Interaction (TOCHI) 28, 6 (2021), 1-34.
  108. Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng. 2023. Deep Long-Tailed Learning: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 9 (2023), 10795-10816.

Cited by

Sorted by citations

Authors
Hall, S. M., Dalal, S., Sefala, R., Yuehgoh, F., Alaagib, A., Hamzaoui, I., Ishida, S., Magomere, J., Crais, L., Salama, A., & Afonja, T.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2502.05961
arXiv
2502.05961
OpenAlex
W4407386075
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, other-oa)
  • arXiv (Cornell University)
  • Proceedings of the ACM on Human-Computer Interaction (cc-by)
  • CISPA Helmholtz Center (cc-by)
Funders
European Commission (101070617)
Topics
  • Social Sciences › Arts and Humanities › Literature and Literary Theory › Digital Humanities and Scholarship
  • Life Sciences › Agricultural and Biological Sciences › Food Science › Culinary Culture and Tourism
  • Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
Keywords
data work, cultural diversity, human labor, cultural data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review