CoChat
Sign Up
Cited by 17Ethics and Social Impacts of AIInnovative Human-Technology InteractionInformation Systems Theories and Implementation

From Fitting Participation to Forging Relationships: The Art of Participatory ML

Open access
Cooper, Ned, Zafiroglu, Alexandra·2024·Proceedings of the CHI Conference on Human Factors in Computing Systems
DOI 10.1145/3613904.3642775
Open this paper in CoChat — free account

Abstract

Participatory machine learning (ML) encourages the inclusion of end users and people affected by ML systems in design and development processes. We interviewed 18 participation brokers—individuals who facilitate such inclusion and transform the products of participants’ labour into inputs for an ML artefact or system—across a range of organisational settings and project locations. Our findings demonstrate the inherent challenges of integrating messy contextual information generated through participation with the structured data formats required by ML workflows and the uneven power dynamics in project contexts. We advocate for evolution in the role of brokers to more equitably balance value generated in Participatory ML projects for design and development teams with value created for participants. To move beyond ‘fitting’ participation to existing processes and empower participants to envision alternative futures through ML, brokers must become educators and advocates for end users, while attending to frustration and dissent from indirect stakeholders.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3613904.3642775

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Delgado et al. found that most AI projects bring stakeholders merely in as consultants, in one-off preference elicitation or UI decisions.

    “However, in a recent review paper, Delgado et al. [31] found that most AI projects bring stakeholders merely in as consultants, in one-off preference elicitation or UI decisions (cf. also [27, 108]).”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [27]

    Section 2.3 (p. 2) reports a tendency in participatory ML projects to inform or consult participants rather than form equitable partnerships, citing Delgado et al. for that point. But this source is Cooper et al.’s interview study, not a review by Delgado et al.; it does not establish that most AI projects use one-off consultations or limit stakeholders to UI decisions. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • There is a growing trend in proxy-based participation, whereby individuals familiar with a stakeholder community, including UX/HCI practitioners, often 'stand-in' for others rather than enabling communities to speak for themselves.

    “There is also a growing trend in proxy-based participation, whereby individuals familiar with a stakeholder community, including UX/HCI practitioners, often 'stand-in' for others [27, 31, 106] rather than enabling communities to speak for themselves.”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [27]

    Section 4.1 (p. 4) describes brokers who sit between ML systems and affected people, and reports difficulty engaging indirect stakeholders. But the paper does not establish a growing trend of UX/HCI practitioners or other community-familiar individuals standing in for communities rather than enabling them to speak for themselves. Its discussion of proxies in Section 2.3 (p. 2) concerns algorithmic proxies, not those individuals. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 56

  1. [1][1] Kars Alfrink, Ianus Keller, Neelke Doorn, and Gerd Kortuem. 2023. Contestable Camera Cars: A Speculative Design Exploration of Public AI That Is Open and Responsive to Dispute. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI '23, Article 8) . Association for Computing Machinery, New York, NY, USA, 1-16. -
  2. [3][3] Sherry R Arnstein. 1969. A Ladder Of Citizen Participation. Journal of the American Institute of Planners 35, 4 (July 1969), 216-224. -
  3. [5][5] Ruha Benjamin. 2019. Race After Technology: Abolitionist Tools for the New Jim Code . Polity Press, Cambridge, UK. -
  4. [8][8] Erling Björgvinsson, Pelle Ehn, and Per-Anders Hillgren. 2010. Participatory Design and 'Democratizing Innovation'. In Proceedings of the 11th Biennial Participatory Design Conference (Sydney, Australia) (PDC '10) . Association for Computing Machinery, New York, NY, USA, 41-50. -
  5. [9][9] Susanne Bødker. 2006. When second wave HCI meets third wave challenges. In Proceedings of the 4th Nordic Conference on Human-Computer Interaction: Changing Roles (Oslo, Norway) (NordiCHI '06) . Association for Computing Machinery, New York, NY, USA, 1-8. -
  6. [10][10] Susanne Bødker and Morten Kyng. 2018. Participatory Design That MattersFacing the Big Issues. ACM Transactions on Computer-Human Interaction 25, 1 (Feb. 2018), 1-31. -
  7. [11][11] Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. Man Is to Computer Programmer as Woman Is to Homemaker? Debiasing Word Embeddings. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS'16) . Curran Associates Inc., Red Hook, NY, USA, 4356-4364. -
  8. [12][12] Elizabeth Bondi, Lily Xu, Diana Acosta-Navas, and Jackson A Killian. 2021. Envisioning Communities: A Participatory Approach towards AI for Social Good. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21) . Association for Computing Machinery, New York, NY, USA, 425-436. -
  9. [13][13] Geoffrey C Bowker and Susan Leigh Star. 2000. Sorting Things Out: Classification and Its Consequences . MIT Press, Cambridge, MA, USA. -
  10. [14][14] Tone Bratteteig and Ina Wagner. 2014. Disentangling Participation: Power and Decision-making in Participatory Design . Springer, Switzerland. -
  11. [18][18] Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research, Vol. 81) , Sorelle A Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 77-91. -
  12. [20][20] Paul Chatterton, Duncan Fuller, and Paul Routledge. 2007. Relating Action to Activism: Theoretical and Methodological Reflections. In Participatory Action Research Approaches and Methods: Connecting People, Participation and Place , Sara Kindon, Rachel Pain, and Mike Kesby (Eds.). Routledge, London, UK, 216-222. https://play.google.com/store/books/details?id=daKkkt0NiPQC -
  13. [21][21] Hao-Fei Cheng, Logan Stapleton, Ruiqi Wang, Paige Bullock, Alexandra Chouldechova, Zhiwei Steven Steven Wu, and Haiyi Zhu. 2021. Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . Association for Computing Machinery, New York, NY, USA. -
  14. [22][22] A Feder Cooper, Emanuel Moss, Benjamin Laufer, and Helen Nissenbaum. 2022. Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT '22) . Association for Computing Machinery, New York, NY, USA, 864-876. -
  15. [23][23] Ned Cooper, Tiffanie Horne, Gillian R Hayes, Courtney Heldreth, Michal Lahav, Jess Holbrook, and Lauren Wilcox. 2022. A Systematic Review and Thematic Analysis of Community-Collaborative Approaches to Computing Research. In CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI '22, Article 73) . Association for Computing Machinery, New York, NY, USA, 1-18. -
  16. [24][24] Eric Corbett, Emily Denton, and Sheena Erete. 2023. Power and Public Participation in AI. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (Boston, MA, USA) (EAAMO '23, Article 8) . Association for Computing Machinery, New York, NY, USA, 1-13. https://doi.org/10.1145/3617694.3623228 - · DOI
  17. [25][25] Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2021. Stakeholder Participation in AI: Beyond 'Add Diverse Stakeholders and Stir'. arXiv:2111.01122 [cs.AI] - · arXiv
  18. [26][26] Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023. The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (Boston, MA, USA) (EAAMO '23, Article 37) . Association for Computing Machinery, New York, NY, USA, 1-23. https://doi.org/10.1145/3617694.3623261 - · DOI · Paper page
  19. [27][27] Wesley Hanwen Deng, Boyuan Guo, Alicia Devrio, Hong Shen, Motahhare Eslami, and Kenneth Holstein. 2023. Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI '23, Article 377) . Association for Computing Machinery, New York, NY, USA, 1-18. -
  20. [28][28] Catherine D'Ignazio and Lauren F Klein. 2020. Data Feminism . The MIT Press, Cambridge, MA, USA. -
  21. [29][29] Carl DiSalvo, Andrew Clement, and Volkmar Pipek. 2013. Communities: Participatory Design for, with and by Communities. In Routledge International Handbook of Participatory Design , Jesper Simonsen and Toni Robertson (Eds.). Taylor & Francis Group, London, UK. -
  22. [30][30] Roel Dobbe, Thomas Krendl Gilbert, and Yonatan Mintz. 2021. Hard Choices in Artificial Intelligence. Artificial Intelligence 300 (2021), 103555. -
  23. [31][31] Markus D Dubber, Frank Pasquale, and Sunit Das. 2020. Oxford Handbook of Ethics of AI . Oxford University Press, Oxford, UK. -
  24. [33][33] Michael Feffer, Michael Skirpan, Zachary Lipton, and Hoda Heidari. 2023. From Preference Elicitation to Participatory ML: A Critical Survey & Guidelines for Future Research. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (Montréal , QC, Canada) (AIES '23) . Association for Computing Machinery, New York, NY, USA, 38-48. https://doi.org/10.1145/3600211.3604661 - · DOI
  25. [35][35] Batya Friedman and David G Hendry. 2019. Value Sensitive Design: Shaping Technology with Moral Imagination . The MIT Press, Cambridge, MA, USA. -
  26. [36][36] Ben Gansky and Sean McDonald. 2022. CounterFAccTual: How FAccT Undermines Its Organizing Principles. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT '22) . Association for Computing Machinery, New York, NY, USA, 1982-1992. -
  27. [40][40] Judith Gregory. 2003. Scandinavian approaches to participatory design. International Journal of Engineering Education 19, 1 (2003), 62-74. -
  28. [41][41] Lara Groves, Aidan Peppin, Andrew Strait, and Jenny Brennan. 2023. Going public: the role of public participation approaches in commercial AI labs. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Chicago, IL, USA) (FAccT '23) . Association for Computing Machinery, New York, NY, USA, 1162-1173. -
  29. [42][42] Aaron Halfaker and R Stuart Geiger. 2020. ORES: Lowering Barriers with Participatory Machine Learning in Wikipedia. Proceedings of the ACM on HumanComputer Interaction 4, CSCW2 (Oct. 2020), 1-37. -
  30. [47][47] Michael Katell, Meg Young, Dharma Dailey, Bernease Herman, Vivian Guetler, Aaron Tam, Corinne Bintz, Daniella Raz, and P M Krafft. 2020. Toward situated interventions for algorithmic equity: lessons from the field. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* '20) . Association for Computing Machinery, New York, NY, USA, 45-55. -
  31. [51][51] Min Kyung Lee, Daniel Kusbit, Anson Kahng, Ji Tae Kim, Xinran Yuan, Allissa Chan, Daniel See, Ritesh Noothigattu, Siheon Lee, Alexandros Psomas, and Ariel D Procaccia. 2019. WeBuildAI: Participatory Framework for Algorithmic Governance. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (Nov. 2019), 1-35. -
  32. [54][54] Michael J Muller and Allison Druin. 2012. Participatory Design: The Third Space in Human-Computer Interaction. In The Human-Computer Interaction Handbook . CRC Press, Boca Raton, FL, USA, 1125-1153. -
  33. [55][55] Michael J Muller and Sarah Kuhn. 1993. Participatory Design. Commun. ACM 36, 6 (June 1993), 24-28. -
  34. [56][56] Laura Nader. 1972. Up the Anthropologist: Perspectives Gained From Studying Up. In Reinventing Anthropology , Dell Hymes (Ed.). Pantheon Books, Random House, New York. -
  35. [57][57] Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila, Timi Fasubaa, Taiwo Fagbohungbe, Solomon Oluwole Akinola, Shamsuddeen Muhammad, Salomon Kabongo Kabenamualu, Salomey Osei, Freshia Sackey, Rubungo Andre Niyongabo, Ricky Macharm, Perez Ogayo, Orevaoghene Ahia, Musie Meressa Berhe, Mofetoluwa Adeyemi, Masabata Mokgesi-Selinga, Lawrence Okegbemi, Laura Martinus, Kolawole Tajudeen, Kevin Degila, Kelechi Ogueji, Kathleen Siminyu, Julia Kreutzer, Jason Webster, Jamiil Toure Ali, Jade Abbott, Iroro Orife, Ignatius Ezeani, Idris Abdulkadir Dangana, Herman Kamper, Hady Elsahar, Goodness Duru, Ghollah Kioko, Murhabazi Espoir, Elan van Biljon, Daniel Whitenack, Christopher Onyefuluchi, Chris Chinenye Emezue, Bonaventure F P Dossou, Blessing Sibanda, Blessing Bassey, Ayodele Olabiyi, Arshath Ramkilowan, Alp Öktem, Adewale Akinfaderin, and Abdallah Bashir. 2020. Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages. In Findings of the Association for Computational Linguistics: EMNLP 2020 . Association for Computational Linguistics, Online, 2144-2160. https://doi.org/10.18653/v1/2020.findings-emnlp.195 - · DOI
  36. [58][58] Safiya Umoja Noble. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism . NYU Press, New York, NY, USA. -
  37. [60][60] Desmond U Patton, Fei-Tzin Lee, William R Frey, Kathleen McKeown, Kyle A McGregor, and Emanuel Moss. 2020. Contextual Analysis of Social Media: The Promise and Challenge of Eliciting Context in Social Media Posts with Natural Language Processing. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 2020 (Feb. 2020), 337-342. -
  38. [61][61] Vinodkumar Prabhakaran and Donald Martin, Jr. 2020. Participatory Machine Learning Using Community-Based System Dynamics. Health and Human Rights Journal 22, 2 (Dec. 2020), 71-74. -
  39. [62][62] Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson. 2022. Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT '22) . Association for Computing Machinery, New York, NY, USA, 1776-1826. https://doi.org/10.1145/3531146.3533231 - · DOI
  40. [63][63] Organizers Of Queerinai, Anaelia Ovalle, Arjun Subramonian, Ashwin Singh, Claas Voelcker, Danica J Sutherland, Davide Locatelli, Eva Breznik, Filip Klubicka, Hang Yuan, Hetvi J, Huan Zhang, Jaidev Shriram, Kruno Lehman, Luca Soldaini, Maarten Sap, Marc Peter Deisenroth, Maria Leonor Pacheco, Maria Ryskina, Martin Mundt, Milind Agarwal, Nyx Mclean, Pan Xu, A Pranav, Raj Korpan, Ruchira Ray, Sarah Mathew, Sarthak Arora, St John, Tanvi Anand, Vishakha Agrawal, William Agnew, Yanan Long, Zijie J Wang, Zeerak Talat, Avijit Ghosh, Nathaniel Dennler, Michael Noseworthy, Sharvani Jha, Emi Baylor, Aditya Joshi, Natalia Y Bilenko, Andrew Mcnamara, Raphael Gontijo-Lopes, Alex Markham, Evyn Dong, Jackie Kay, Manu Saraswat, Nikhil Vytla, and Luke Stark. 2023. Queer In AI: A Case Study in Community-Led Participatory AI. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Chicago, IL, USA) (FAccT '23) . Association for Computing Machinery, New York, NY, USA, 1882-1895. https://doi.org/10.1145/3593013.3594134 - · DOI
  41. [64][64] Samantha Robertson and Niloufar Salehi. 2020. What If I Don't Like Any Of The Choices? The Limits of Preference Elicitation for Participatory Algorithm Design. arXiv:2007.06718 [cs.CY] - · arXiv
  42. [65][65] Nithya Sambasivan and Rajesh Veeraraghavan. 2022. The Deskilling of Domain Expertise in AI Development. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI '22, Article 587) . Association for Computing Machinery, New York, NY, USA, 1-14. -
  43. [66][66] Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2021. Gender Bias in Machine Translation. Transactions of the Association for Computational Linguistics (TACL) 9 (Aug. 2021), 845-874. -
  44. [67][67] Douglas Schuler and Aki Namioka. 1993. Participatory Design: Principles and Practices . CRC Press, Boca Raton, FL, USA. -
  45. [68][68] Andrew D Selbst, Danah Boyd, Sorelle A Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. Fairness and Abstraction in Sociotechnical Systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency (Atlanta, GA, USA) (FAT* '19) . Association for Computing Machinery, New York, NY, USA, 59-68. -
  46. [69][69] Ranjit Singh, Rigoberto Lara Guzmán, and Patrick Davison. 2022. Parables of AI in/from the Majority World. (Dec. 2022). https://doi.org/10.2139/ssrn.4258527 - · DOI
  47. [71][71] Yngve Sundblad. 2011. UTOPIA: Participatory Design from Scandinavia to the World. In History of Nordic Computing 3 , John Impagliazzo, Per Lundin, and Benkt Wangler (Eds.). Springer, Berlin, Germany; London, UK, 176-186. -
  48. [72][72] Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxen, Angeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D'Ignazio. 2022. Towards Intersectional Feminist and Participatory ML: A Case Study in Supporting Feminicide Counterdata Collection. In 2022 ACM Conference on Fairness, Accountability, and Transparency . Association for Computing Machinery, Seoul, Republic of Korea, 667-678. -
  49. [74][74] Pranav Narayanan Venkit, Mukund Srinath, and Shomir Wilson. 2022. A Study of Implicit Bias in Pretrained Language Models against People with Disabilities. In Proceedings of the 29th International Conference on Computational Linguistics . International Committee on Computational Linguistics, Gyeongju, Republic of Korea, 1324-1332. -
  50. [75][75] John Vines, Rachel Clarke, Peter Wright, John McCarthy, and Patrick Olivier. 2013. Configuring participation: on how we involve people in design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI '13) . Association for Computing Machinery, New York, NY, USA, 429-438. -
  51. [76][76] Carol A B Warren. 2001. Qualitative Interviewing. In Handbook of Interview Research . SAGE Publications Ltd, Thousand Oaks, California, USA, 83-102. -
  52. [78][78] Angie Zhang, Olympia Walker, Kaci Nguyen, Jiajun Dai, Anqing Chen, and Min Kyung Lee. 2023. Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (April 2023), 1-32. -
  53. [79][79] Haiyi Zhu, Bowen Yu, Aaron Halfaker, and Loren Terveen. 2018. Value-Sensitive Algorithm Design: Method, Case Study, and Lessons. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (Nov. 2018), 1-23. -
  54. [80][80] Malte Ziewitz and Ranjit Singh. 2021. Critical Companionship: Some Sensibilities for Studying the Lived Experience of Data Subjects. Big Data & Society 8, 2 (July 2021), 20539517211061122. https://doi.org/10.1177/20539517211061122 - · DOI
  55. [81][81] Jonathan Zong and J Nathan Matias. 2023. Data Refusal From Below: A Framework for Understanding, Evaluating, and Envisioning Refusal as Design. ACM Journal on Responsible Computing (Oct. 2023). https://doi.org/10.1145/3630107 - · DOI
  56. [83][83] Douglas Zytko, Pamela J. Wisniewski, Shion Guha, Eric P. S. Baumer, and Min Kyung Lee. 2022. Participatory Design of AI Systems: Opportunities and Challenges Across Diverse Users, Relationships, and Application Domains. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA '22, Article 154) . Association for Computing Machinery, New York, NY, USA, 1-4.

Cited by

Sorted by citations

Authors
Cooper, N., & Zafiroglu, A.
Affiliations
Australian National University (AU)
ORCID
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-9
DOI
10.1145/3613904.3642775
OpenAlex
W4392735893
Cited by
17
References
9
Citations by year
2026: 3 · 2025: 11 · 2024: 3
FWCI
6.20
Type
conference paper
Language
en
Open access
Licence
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • ANU Open Research (Australian National University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
Australian Government
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
Keywords
end-user involvement, power relations, stakeholder engagement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review