The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
WeBuildAI: Participatory Framework for Algorithmic Governance
Open accessAbstract
Algorithms increasingly govern societal functions, impacting multiple stakeholders and social groups. How can we design these algorithms to balance varying interests in a moral, legitimate way? As one answer to this question, we present WeBuildAI, a collective participatory framework that enables people to build algorithmic policy for their communities. The key idea of the framework is to enable stakeholders to construct a computational model that represents their views and to have those models vote on their behalf to create algorithmic policy. As a case study, we applied this framework to a matching algorithm that operates an on-demand food donation transportation service in order to adjudicate equity and efficiency trade-offs. The service's stakeholders--donors, volunteers, recipient organizations, and nonprofit employees--used the framework to design the algorithm through a series of studies in which we researched their experiences. Our findings suggest that the framework successfully enabled participants to build models that they felt confident represented their own beliefs. Participatory algorithm design also improved both procedural fairness and the distributive outcomes of the algorithm, raised participants' algorithmic awareness, and helped identify inconsistencies in human decision-making in the governing organization. Our work demonstrates the feasibility, potential and challenges of community involvement in algorithm design.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Construal-level theory of psychological distance.
Trope, Y., & Liberman, N. (2010) · Psychological Review · doi:10.1037/a0018963 · Cited by 6,618
The Social Psychology of Procedural Justice
Lind, E. A., & Tyler, T. R. (1988) · Critical Issues in Social Justice · doi:10.1007/978-1-4899-2115-4 · Cited by 6,585
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836
Gillespie, T. (2010) · New Media & Society · doi:10.1177/1461444809342738 · Cited by 2,522 · cited 1×
“…information, which influences the costs and benefits of their services for their users, news sources, advertisers, and the platforms themselves [39]; on-demand work platforms use algorithms to assign tasks, which affects their customers, their workers, and their own profits [41, 51, 72]; and…”
BackgroundFair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338
Self-citationPutting the public back into governance: The challenges of citizen participation and its future
Fung, A. (2015) · Public Administration Review · doi:10.1111/puar.12361 · Cited by 1,232
Friedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1×
“Fairness is an important value in governing algorithms as algorithms can perpetuate unfair treatment of different populations or stakeholders [27, 35, 85].”
BackgroundIndividual choice behavior: A theoretical analysis.
Luce, R. D. (2005) · doi:10.1037/14396-000 · Cited by 1,157
Plackett, R. L. (1975) · Applied Statistics · doi:10.2307/2346567 · Cited by 731
Research works printed but not linked: 63
- 10.1145/3173574.3173860 · DOI
- 10.1215/9780822387534 · DOI
- Julia Angwin , Jeff Larson , Surya Mattu , and Lauren Kirchner . 2016. Machine bias. ProPublica ( 2016 ). Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica (2016).
- Yukiko Asada . 2005. Assessment of the health of Americans: the average health-related quality of life and its inequality across individuals and groups. Population Health Metrics 3, 1 ( 2005 ), article 7. Yukiko Asada. 2005. Assessment of the health of Americans: the average health-related quality of life and its inequality across individuals and groups. Population Health Metrics 3, 1 (2005), article 7.
- 10.1145/1978942.1979397 · DOI
- Albert Bandura . 2010. Self-efficacy. The Corsini Encyclopedia of Psychology ( 2010 ), 1--3. Albert Bandura. 2010. Self-efficacy. The Corsini Encyclopedia of Psychology (2010), 1--3.
- 10.1509/jmkg.67.1.14.18592 · DOI · Paper page
- 10.1287/mnsc.1120.1549 · DOI · Paper page
- Reuben Binns . 2017. Fairness in machine learning: Lessons from political philosophy. arXiv preprint arXiv:1712.03586 ( 2017 ). Reuben Binns. 2017. Fairness in machine learning: Lessons from political philosophy. arXiv preprint arXiv:1712.03586 (2017). · arXiv
- Keld Bødker , Finn Kensing , and Jesper Simonsen . 2009. Participatory IT Design: Designing for Business and Workplace Realities . MIT press . Keld Bødker, Finn Kensing, and Jesper Simonsen. 2009. Participatory IT Design: Designing for Business and Workplace Realities. MIT press.
- Jean-François Bonnefon , Azim Shariff , and Iyad Rahwan . 2016. The social dilemma of autonomous vehicles. Science 352, 6293 ( 2016 ), 1573--1576. Jean-François Bonnefon, Azim Shariff, and Iyad Rahwan. 2016. The social dilemma of autonomous vehicles. Science 352, 6293 (2016), 1573--1576.
- 10.1145/2207676.2208560 · DOI · Paper page
- Bianca Bosker . 2016. The Binge Breaker. The Atlantic ( 2016 ). Bianca Bosker. 2016. The Binge Breaker. The Atlantic (2016).
- Felix Brandt , Vincent Conitzer , Ulle Endriss , Jérôme Lang , and Ariel D Procaccia . 2016. Handbook of Computational Social Choice . Cambridge University Press . Felix Brandt, Vincent Conitzer, Ulle Endriss, Jérôme Lang, and Ariel D Procaccia. 2016. Handbook of Computational Social Choice. Cambridge University Press.
- 10.1145/3290605.3300271 · DOI · Paper page
- Bill Buxton. 2010. Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann. Bill Buxton. 2010. Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann.
- Anupam Chander . 2016 . The racist algorithm ? Michigan Law Review 115 (2016), 1023 . Anupam Chander. 2016. The racist algorithm? Michigan Law Review 115 (2016), 1023.
- 10.5465/amr.1988.4306983 · doi:10.5465/amr.1988.4306983 · Paper page
- Juliet Corbin , Anselm Strauss , and Anselm L Strauss . 2014. Basics of Qualitative Research . Sage . Juliet Corbin, Anselm Strauss, and Anselm L Strauss. 2014. Basics of Qualitative Research. Sage.
- 10.1037/a0030931 · DOI · Paper page
- John Danaher , Michael J Hogan , Chris Noone , Rónán Kennedy , Anthony Behan , Aisling De Paor , Heike Felzmann, Muki Haklay, Su-Ming Khoo, John Morison, et al. 2017 . Algorithmic governance: Developing a research agenda through the power of collective intelligence. Big Data & Society 4, 2 (2017). John Danaher, Michael J Hogan, Chris Noone, Rónán Kennedy, Anthony Behan, Aisling De Paor, Heike Felzmann, Muki Haklay, Su-Ming Khoo, John Morison, et al. 2017. Algorithmic governance: Developing a research agenda through the power of collective intelligence. Big Data & Society 4, 2 (2017).
- Norman Daniels . 2016. Reflective Equilibrium . In Stanford Encyclopedia of Philosophy. Norman Daniels. 2016. Reflective Equilibrium. In Stanford Encyclopedia of Philosophy.
- 10.1037/h0037613 · DOI · Paper page
- Carl DiSalvo , Illah Nourbakhsh , David Holstius , Ayça Akin , and Marti Louw . 2008 . The Neighborhood Networks project: A case study of critical engagement and creative expression through participatory design . In Proceedings of the 10th Anniversary Conference on Participatory Design. 41--50 . Carl DiSalvo, Illah Nourbakhsh, David Holstius, Ayça Akin, and Marti Louw. 2008. The Neighborhood Networks project: A case study of critical engagement and creative expression through participatory design. In Proceedings of the 10th Anniversary Conference on Participatory Design. 41--50.
- 10.1145/2090236.2090255 · DOI · Paper page
- FATML. 2018. Fairness Accountability and Transparency in Machine Learning Workshop. FATML. 2018. Fairness Accountability and Transparency in Machine Learning Workshop.
- 10.1111/1467-9760.00181 · DOI · Paper page
- Archon Fung . 2006. Varieties of participation in complex governance. Public administration review 66 ( 2006 ), 66--75. Archon Fung. 2006. Varieties of participation in complex governance. Public administration review 66 (2006), 66--75.
- 10.2307/2223319 · DOI · Paper page
- Mary L Gray and Siddharth Suri . 2019 . Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass . Eamon Dolan Books . Mary L Gray and Siddharth Suri. 2019. Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass. Eamon Dolan Books.
- 10.1080/01972240050133634 · DOI
- 10.1145/775047.775067 · DOI · Paper page
- Anson Kahng , Min Kyung Lee , Ritesh Noothigattu , Ariel D Procaccia , and Christos-Alexandros Psomas . 2019 . Statistical Foundations of Virtual Democracy . In Proceedings of the 36th International Conference on Machine Learning. 3173--3182 . Anson Kahng, Min Kyung Lee, Ritesh Noothigattu, Ariel D Procaccia, and Christos-Alexandros Psomas. 2019. Statistical Foundations of Virtual Democracy. In Proceedings of the 36th International Conference on Machine Learning. 3173--3182.
- 10.1080/1369118X.2016.1154087 · DOI · Paper page
- Jon Kleinberg , Sendhil Mullainathan , and Manish Raghavan . 2016. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807 ( 2016 ). Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2016. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807 (2016). · arXiv
- 10.1145/3359284 · DOI · Paper page
- 10.1145/3025453.3025884 · DOI
- 10.1145/2702123.2702548 · DOI · Paper page
- 10.1007/BF00133443 · DOI · Paper page
- 10.1145/3173574.3173583 · DOI
- Jessica Morley and Luciano Floridi . 2019. The Limits of Empowerment: How to Reframe the Role of mHealth Tools in the Healthcare Ecosystem. Science and Engineering Ethics ( 2019 ), 1--25. Jessica Morley and Luciano Floridi. 2019. The Limits of Empowerment: How to Reframe the Role of mHealth Tools in the Healthcare Ecosystem. Science and Engineering Ethics (2019), 1--25.
- Frederick Mosteller . 2006. Remarks on the method of paired comparisons: I. The least squares solution assuming equal standard deviations and equal correlations . In Selected Papers of Frederick Mosteller . Springer , 157--162. Frederick Mosteller. 2006. Remarks on the method of paired comparisons: I. The least squares solution assuming equal standard deviations and equal correlations. In Selected Papers of Frederick Mosteller. Springer, 157--162.
- Michael J Muller . 2009. Participatory design: The third space in HCI . In Human-Computer Interaction . CRC press , 181--202. Michael J Muller. 2009. Participatory design: The third space in HCI. In Human-Computer Interaction. CRC press, 181--202.
- Ritesh Noothigattu , Neil S Gaikwad , Edmond Awad , Sohan Dsouza , Iyad Rahwan , Pradeep Ravikumar , and Ariel D Procaccia . 2018 . A Voting-Based System for Ethical Decision Making . In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI). Ritesh Noothigattu, Neil S Gaikwad, Edmond Awad, Sohan Dsouza, Iyad Rahwan, Pradeep Ravikumar, and Ariel D Procaccia. 2018. A Voting-Based System for Ethical Decision Making. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI).
- Arthur M Okun . 1975. Equality and Efficiency: The Big Tradeoff . Brookings Institution Press . Arthur M Okun. 1975. Equality and Efficiency: The Big Tradeoff. Brookings Institution Press.
- Michael Q Patton . 1980. Qualitative Research and Evaluation Methods . Sage . Michael Q Patton. 1980. Qualitative Research and Evaluation Methods. Sage.
- 10.1007/s10676-017-9430-8 · DOI · Paper page
- 10.2307/1600311 · DOI · Paper page
- John Rawls . 2009. A Theory of Justice . Harvard University Press . John Rawls. 2009. A Theory of Justice. Harvard University Press.
- Dillon Reisman , Jason Schultz , K Crawford , and M Whittaker . 2018. Algorithmic impact assessments: A practical framework for public agency accountability . AI Now Institute . Dillon Reisman, Jason Schultz, K Crawford, and M Whittaker. 2018. Algorithmic impact assessments: A practical framework for public agency accountability. AI Now Institute.
- 10.1145/2699742 · DOI
- 10.1146/annurev.soc.28.110601.141114 · DOI · Paper page
- Eric Schwitzgebel . 2019. Belief . In The Stanford Encyclopedia of Philosophy (summer 2019 ed.), Edward N . Zalta (Ed.). Metaphysics Research Lab, Stanford University . Eric Schwitzgebel. 2019. Belief. In The Stanford Encyclopedia of Philosophy (summer 2019 ed.), Edward N. Zalta (Ed.). Metaphysics Research Lab, Stanford University.
- Amartya Sen . 2017. Collective Choice and Social Welfare: Expanded edition . Penguin UK. Amartya Sen. 2017. Collective Choice and Social Welfare: Expanded edition. Penguin UK.
- 10.1145/3134732 · DOI
- Richard H Thaler Cass R Sunstein and John P Balz. 2014. Choice architecture. Manuscript. Richard H Thaler Cass R Sunstein and John P Balz. 2014. Choice architecture. Manuscript. · DOI
- Louis L Thurstone . 1959. The Measurement of Values . University of Chicago Press. Louis L Thurstone. 1959. The Measurement of Values. University of Chicago Press.
- 10.1145/3137133.3137163 · doi:10.1145/3137133.3137163
- 10.1145/2470654.2470716 · DOI · Paper page
- Max Weber . 2009. The Theory of Social and Economic Organization . Simon and Schuster . Max Weber. 2009. The Theory of Social and Economic Organization. Simon and Schuster.
- Meredith Whittaker , Kate Crawford , Roel Dobbe , Genevieve Fried , Elizabeth Kaziunas , Varoon Mathur , and Jason Schultz . 2018. AI Now Report 2018 . Meredith Whittaker, Kate Crawford, Roel Dobbe, Genevieve Fried, Elizabeth Kaziunas, Varoon Mathur, and Jason Schultz. 2018. AI Now Report 2018.
- Langdon Winner . 1980. Do artifacts have politics? Daedalus ( 1980 ), 121--136. Langdon Winner. 1980. Do artifacts have politics? Daedalus (1980), 121--136.
- 10.1111/rssa.12227 · DOI · Paper page
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationExpanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538
Algorithmic management in a work context
Jarrahi, M. H., Newlands, G., Lee, M. K., Wolf, C. T., Kinder, E., & Sutherland, W. (2021) · Big Data & Society · doi:10.1177/20539517211020332 · Cited by 532
Self-citationNg, D. T. K., Wu, W., Leung, J. K. L., Chiu, T. K. F., & Chu, S. K. W. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13411 · Cited by 390
Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literature
Starke, C., Baleis, J., Keller, B., & Marcinkowski, F. (2022) · Big Data & Society · doi:10.1177/20539517221115189 · Cited by 316
Metadata
- Authors
- Lee, M. K., Kusbit, D., Kahng, A., Kim, J. T., Yuan, X., Chan, A., See, D., Noothigattu, R., Lee, S., Psomas, A., & Procaccia, A. D.
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2019-11-07
- Volume
- 3
- Issue
- CSCW
- Pages
- 1-35
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W2982915530
- Cited by
- 206
- References
- 21
- FWCI
- 32.33
- Type
- journal article
- Language
- en
- Open access
- bronze
- Funders
- Uptake & CMU Machine Learning for Social Good Grant
- National Science Foundation (1651566)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: WeBuildAI: Participatory Framework for Algorithmic Governance. https://app.cochat.ai/papers/10.1145/3359283
- The paper
- Lee, M. K., Kusbit, D., Kahng, A., Kim, J. T., Yuan, X., Chan, A., See, D., Noothigattu, R., Lee, S., Psomas, A., & Procaccia, A. D. (2019). WeBuildAI: Participatory Framework for Algorithmic Governance. Proceedings of the ACM on Human-Computer Interaction. https://doi.org/10.1145/3359283
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.