CoChat
Sign Up
Cited by 330Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Smart Cities and Technologies

Power to the People? Opportunities and Challenges for Participatory AI

Open access
Birhane, Abeba, Isaac, William, Prabhakaran, Vinodkumar, Diaz, Mark, Elish, Madeleine Clare, Gabriel, Iason, Mohamed, Shakir·2022·Equity and Access in Algorithms Mechanisms and Optimization
DOI 10.1145/3551624.3555290
Open this paper in CoChat — free accountOpen PDF

Abstract

Participatory approaches to artificial intelligence (AI) and machine learning (ML) are gaining momentum: the increased attention comes partly with the view that participation opens the gateway to an inclusive, equitable, robust, responsible and trustworthy AI. Among other benefits, participatory approaches are essential to understanding and adequately representing the needs, desires and perspectives of historically marginalized communities. However, there currently exists lack of clarity on what meaningful participation entails and what it is expected to do. In this paper we first review participatory approaches as situated in historical contexts as well as participatory methods and practices within the AI and ML pipeline. We then introduce three case studies in participatory AI. Participation holds the potential for beneficial, emancipatory and empowering technology design, development and deployment while also being at risk for concerns such as cooptation and conflation with other activities. We lay out these limitations and concerns and argue that as participatory AI/ML becomes in vogue, a contextual and nuanced understanding of the term as well as consideration of who the primary beneficiaries of participatory activities ought to be constitute crucial factors to realizing the benefits and opportunities that participation brings.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3551624.3555290

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Communities most in need of public services are also most at-risk of being disadvantaged by AI systems.

    “Further, in many instances, the communities most in need of public services are also the communities that are most at-risk of being disadvantaged by AI systems [15, 17, 33, 95, 100, 102].”

    From “It’s Just a Wild, Wild West”: Harnessing Public Procurement as an AI Governance Mechanism (2026) · cites this paper as [15]

    Section 3.2 (Case 2: Fighting for Māori data rights; page unavailable) says people disproportionately affected by oppressive social structures often bear the negative impacts of technology. That supports the concern about unequal risk, but the paper does not identify those people as the communities most in need of public services or establish the claimed overlap. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Critical scholars highlight that RAI frameworks often reflect Western-centric values, overlooking the specific needs and cultural contexts of the Global South.

    “However, critical scholars highlight that these frameworks often reflect Western-centric values, overlooking the specific needs and cultural contexts of the Global South [1, 3, 10, 16].”

    From Designing Responsible AI with Indigenous Communities: Towards Participatory AI with ovaHimba and San People in Namibia (2026) · cites this paper as [10]

    Section 3.2 (page unavailable) describes how Western-based open-data initiatives conflicted with Māori data sovereignty, and section 3.1 (page unavailable) emphasizes African language communities’ values and needs. These examples support the concern about overlooking local contexts, but the paper does not establish that responsible-AI frameworks generally or often reflect Western-centric values. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Participatory design has been slow in its direct engagement with AI.

    “Although participatory design has been slow in its direct engagement with AI, the focus on democratic technology design and computational empowerment at the community and policy levels has great potential to develop sustainable and decolonial approaches to AI in local global south environments [10, 16, 63].”

    From Designing Responsible AI with Indigenous Communities: Towards Participatory AI with ovaHimba and San People in Namibia (2026) · cites this paper as [10]

  • Participatory design is central to the realisation of RAI in Indigenous contexts, prioritising community participation throughout the AI lifecycle.

    “Participatory design is central to the realisation of RAI in these contexts, which prioritises community participation throughout the AI lifecycle, ensuring that community members actively shape AI systems to reflect their needs, values, and knowledge systems [10, 15].”

    From Designing Responsible AI with Indigenous Communities: Towards Participatory AI with ovaHimba and San People in Namibia (2026) · cites this paper as [10]

  • Other authors call for the enhanced use of participatory approaches to AI and automated systems, increasingly entangled in our everyday lives.

    “In line with this, other authors call for the enhanced use of participatory approaches to AI and automated systems, increasingly entangled in our everyday lives [3, 10, 16].”

    From Designing Responsible AI with Indigenous Communities: Towards Participatory AI with ovaHimba and San People in Namibia (2026) · cites this paper as [10]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 78 · Cited by 330

Research works printed but not linked: 52

  1. Miguel Arana-Catania, Felix-Anselm Van Lier, Rob Procter, Nataliya Tkachenko, Yulan He, Arkaitz Zubiaga, and Maria Liakata. 2021. Citizen participation and machine learning for a better democracy. arXiv preprint arXiv:2103.00508 (2021). · arXiv
  2. Peter M Asaro. 2000. Transforming society by transforming technology: the science and politics of participatory design. Accounting, Management and Infor- mation Technologies 10, 4 (2000), 257-290.
  3. Imon Banerjee, Ananth Reddy Bhimireddy, John L Burns, Leo Anthony Celi, Li- Ching Chen, Ramon Correa, Natalie Dullerud, Marzyeh Ghassemi, Shih-Cheng Huang, Po-Chih Kuo, et al. 2021. Reading Race: AI Recognises Patient's Racial Identity In Medical Images. arXiv preprint arXiv:2107.10356 (2021). · arXiv
  4. Darin Barney, Gabriella Coleman, Christine Ross, Jonathan Sterne, and Tamar Tembeck. 2016. The participatory condition in the digital age. Vol. 51. U of Minnesota Press.
  5. Lyria Bennett Moses and Janet Chan. 2018. Algorithmic prediction in policing: assumptions, evaluation, and accountability. Policing and society 28, 7 (2018), 806-822.
  6. Elizabeth Bondi, Lily Xu, Diana Acosta-Navas, and Jackson A Killian. 2021. En- visioning Communities: A Participatory Approach Towards AI for Social Good. arXiv preprint arXiv:2105.01774 (2021). · arXiv
  7. Yves Cabannes. 2004. Participatory budgeting: a significant contribution to participatory democracy. Environment and urbanization 16, 1 (2004), 27-46.
  8. Robert Chambers. 1994. The origins and practice of participatory rural appraisal. World development 22, 7 (1994), 953-969.
  9. Alan Chan, Chinasa T Okolo, Zachary Terner, and Angelina Wang. 2021. The Limits of Global Inclusion in AI Development. arXiv preprint arXiv:2102.01265 (2021). · arXiv
  10. Jason Chilvers and Matthew Kearnes. 2015. Remaking participation: Science, environment and emergent publics. Routledge.
  11. Nick Couldry and Ulises A Mejias. 2019. The costs of connection. Stanford University Press.
  12. Andy Dearden and Haider Rizvi. 2008. Participatory design and participatory development: a comparative review. (2008).
  13. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248-255.
  14. Emily Denton, Alex Hanna, Razvan Amironesei, Andrew Smart, Hilary Nicole, and Morgan Klaus Scheuerman. 2020. Bringing the people back in: Contesting benchmark machine learning datasets. arXiv preprint arXiv:2007.07399 (2020). · arXiv
  15. Joseph Donia and James A Shaw. 2021. Co-design and ethical artificial intelligence for health: An agenda for critical research and practice. Big Data & Society 8, 2 (2021), 20539517211065248.
  16. Paul Farmer, Fernet Léandre, Joia Mukherjee, Rajesh Gupta, Laura Tarter, and Jim Yong Kim. 2001. Community-based treatment of advanced HIV disease: intro- ducing DOT-HAART (directly observed therapy with highly active antiretroviral therapy). Bulletin of the World Health Organization 79, 12 (2001), 1145.
  17. James Ferguson. 1994. The anti-politics machine:" development, " depoliticization, and bureaucratic power in Lesotho. U of Minnesota Press.
  18. Seth Frey, PM Krafft, and Brian C Keegan. 2019. " This Place Does What It Was Built For" Designing Digital Institutions for Participatory Change. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-31.
  19. Iason Gabriel. 2017. Effective altruism and its critics. Journal of Applied Philosophy 34, 4 (2017), 457-473.
  20. Iason Gabriel. 2022. Toward a Theory of Justice for Artificial Intelligence. Daedalus 151, 2 (2022), 218-231.
  21. R Stuart Geiger, Kevin Yu, Yanlai Yang, Mindy Dai, Jie Qiu, Rebekah Tang, and Jenny Huang. 2020. Garbage in, garbage out? Do machine learning application papers in social computing report where human-labeled training data comes from?. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 325-336.
  22. Aaron Halfaker and R Stuart Geiger. 2020. Ores: Lowering barriers with participa- tory machine learning in wikipedia. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (2020), 1-37.
  23. Christina N Harrington. 2020. The forgotten margins: what is community-based participatory health design telling us? Interactions 27, 3 (2020), 24-29.
  24. Sarah Holland, Ahmed Hosny, Sarah Newman, Joshua Joseph, and Kasia Chmielin- ski. 2018. The dataset nutrition label: A framework to drive higher data quality standards. arXiv preprint arXiv:1805.03677 (2018). · arXiv
  25. Lilly C Irani and M Six Silberman. 2013. Turkopticon: Interrupting worker invisibility in amazon mechanical turk. In Proceedings of the SIGCHI conference on human factors in computing systems. 611-620.
  26. Eun Seo Jo and Timnit Gebru. 2020. Lessons from archives: Strategies for collect- ing sociocultural data in machine learning. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 306-316.
  27. Michael Katell, Meg Young, Dharma Dailey, Bernease Herman, Vivian Guetler, Aaron Tam, Corinne Bintz, Daniella Raz, and P. M. Krafft. 2020. Toward Situated Interventions for Algorithmic Equity: Lessons from the Field. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* '20). Association for Computing Machinery, New York, NY, USA, 45-55.
  28. Christopher M Kelty. 2017. The Participatory Development Toolkit. In Little Development Devices / Humanitarian Goods, Vol. 9. Limn.
  29. Christopher M Kelty. 2020. The participant: A century of participation in four stories. University of Chicago Press.
  30. Philip Kraft and Jørgen P Bansler. 1994. The collective resource approach: the Scandinavian experience. Scandinavian Journal of Information Systems 6, 1 (1994), 4.
  31. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436-444.
  32. Min Kyung Lee, Daniel Kusbit, Anson Kahng, Ji Tae Kim, Xinran Yuan, Allissa Chan, Daniel See, Ritesh Noothigattu, Siheon Lee, Alexandros Psomas, et al. 2019. WeBuildAI: Participatory framework for algorithmic governance. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-35.
  33. Lord Frederick JD Lugard. 1922. The dual mandate in British tropical Africa. Routledge.
  34. Kristian Lum and William Isaac. 2016. To predict and serve? Significance 13, 5 (2016), 14-19.
  35. John Mackenzie, Poh-Ling Tan, Suzanne Hoverman, and Claudia Baldwin. 2012. The value and limitations of participatory action research methodology. Journal of hydrology 474 (2012), 11-21.
  36. Michael Majale. 2008. Employment creation through participatory urban planning and slum upgrading: The case of Kitale, Kenya. Habitat International 32, 2 (2008), 270-282.
  37. Donald Martin Jr, Vinodkumar Prabhakaran, Jill Kuhlberg, Andrew Smart, and William S Isaac. 2020. Participatory problem formulation for fairer machine learn- ing through community based system dynamics. arXiv preprint arXiv:2005.07572 (2020). · arXiv
  38. Timothy Mitchell. 2002. Rule of experts. University of California Press.
  39. Shakir Mohamed, Marie-Therese Png, and William Isaac. 2020. Decolonial AI: Decolonial theory as sociotechnical foresight in artificial intelligence. Philosophy & Technology 33, 4 (2020), 659-684.
  40. Michael J Muller and Allison Druin. 2012. Participatory design: The third space in human-computer interaction. In The Human-Computer Interaction Handbook. CRC Press, 1125-1153.
  41. Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila, Timi Fasubaa, Tajudeen Kolawole, Taiwo Fagbohungbe, Solomon Oluwole Akinola, Shamsud- deen Hassan Muhammad, Salomon Kabongo, Salomey Osei, et al. 2020. Partici- patory research for low-resourced machine translation: A case study in african languages. arXiv preprint arXiv:2010.02353 (2020). · arXiv
  42. Jennifer Pierre, Roderic Crooks, Morgan Currie, Britt Paris, and Irene Pasquetto. 2021. Getting Ourselves Together: Data-centered participatory design research & epistemic burden. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-11.
  43. Kathryn S Quick and Martha S Feldman. 2011. Distinguishing participation and inclusion. Journal of planning education and research 31, 3 (2011), 272-290.
  44. Rashida Richardson, Jason M Schultz, and Kate Crawford. 2019. Dirty data, bad predictions: How civil rights violations impact police data, predictive policing systems, and justice. NYUL Rev. Online 94 (2019), 15.
  45. Sarah T Roberts. 2019. Behind the screen. In Behind the Screen. Yale University Press.
  46. Scott Scheall and Parker Crutchfield. 2021. The priority of the epistemic. Episteme 18, 4 (2021), 726-737.
  47. Andrew D Selbst, Danah Boyd, Sorelle A Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. Fairness and abstraction in sociotechnical systems. In Proceedings of the conference on fairness, accountability, and transparency. 59-68.
  48. Amartya Kumar Sen. 2009. The idea of justice. Harvard University Press.
  49. Gayatri Chakravorty Spivak. 2003. Can the subaltern speak? Die Philosophin 14, 27 (2003), 42-58.
  50. Sabine N van der Veer, Lisa Riste, Sudeh Cheraghi-Sohi, Denham L Phipps, Mary P Tully, Kyle Bozentko, Sarah Atwood, Alex Hubbard, Carl Wiper, Malcolm Oswald, et al. 2021. Trading off accuracy and explainability in AI decision-making: findings from 2 citizens' juries. Journal of the American Medical Informatics Association 28, 10 (2021), 2128-2138.
  51. Jenny Waycott, Frank Vetere, Sonja Pedell, Amee Morgans, Elizabeth Ozanne, and Lars Kulik. 2016. Not for me: Older adults choosing not to participate in a social isolation intervention. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 745-757.
  52. Susanne Weber, Marian Harbach, and Matthew Smith. 2015. Participatory design for security-related user interfaces. Proc. USEC 15 (2015).

Authors
Birhane, A., Isaac, W., Prabhakaran, V., Diaz, M., Elish, M. C., Gabriel, I., & Mohamed, S.
Affiliations
  • University College Dublin (IE)
  • Google DeepMind (United Kingdom) (GB)
  • Google (United States) (US)
  • Google, USA
ORCID
Venue
Equity and Access in Algorithms Mechanisms and Optimization
Publisher
ACM
Published
2022-10-06
Pages
1-8
DOI
10.1145/3551624.3555290
OpenAlex
W4296405185
Cited by
330
References
78
Citations by year
2026: 93 · 2025: 116 · 2024: 72 · 2023: 48 · 2022: 1
FWCI
58.88
Type
conference paper
Language
en
Open access
Licence
Locations
  • ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Engineering › Media Technology › Smart Cities and Technologies
Keywords
participatory AI, meaningful participation, marginalized communities, co-optation, algorithmic fairness, responsible AI, participatory approach

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review