Value-Sensitive Algorithm Design: Method, Case Study, and Lessons
Abstract
Most commonly used approaches to developing automated or artificially intelligent algorithmic systems are Big Data-driven and machine learning-based. However, these approaches can fail, for two notable reasons: (1) they may lack critical engagement with users and other stakeholders; (2) they rely largely on historical human judgments, which do not capture and incorporate human insights into how the world can be improved in the future. We propose and describe a novel method for the design of such algorithms, which we call Value Sensitive Algorithm Design. Value Sensitive Algorithm Design incorporates stakeholders' tacit knowledge and explicit feedback in the early stages of algorithm creation. This increases the chance to avoid biases in design choices or to compromise key stakeholder values. Generally, we believe that algorithms should be designed to balance multiple stakeholders' needs, motivations, and interests, and to help achieve important collective goals. We also describe a specific project "Designing Intelligent Socialization Algorithms for WikiProjects in Wikipedia" to illustrate our method. We intend this paper to contribute to the rich ongoing conversation concerning the use of algorithms in supporting critical decision-making in society.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Trying to satisfy a particular notion of fairness may cause harm in another dimension without the developers realizing it.
“By trying to satisfy a particular notion of fairness (e.g., according to a particular quantitative definition proposed in prior literature), one might actually cause harm in another dimension without realizing it”
From Equity and Artificial Intelligence in Education: Will "AIEd" Amplify or Alleviate Inequities in Education? (2021) · cites this paper as Zhu, Yu, Halfaker, & Terveen, 2018
Cites
Wicked Problems in Design Thinking
Buchanan, R. (1992) · Design Issues · doi:10.2307/1511637 · Cited by 3,539 · cited 1× · see citing works
“We see the Value-Sensitive Algorithm Design method best suited for solving “wicked” problems [4] that lack clear ground truth.”
MethodAlgorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · see citing works
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1× · see citing works
“The second challenge is how to address the fundamental mismatch between human styles of interpretation, reasoning, and inputs and statistical optimizations of high-dimensional data [5].”
BackgroundTowards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230 · cited 1× · see citing works
“We measured the effect of our system on new editors, specifically in terms of the number of their edits within the relevant WikiProjects and in the rest of Wikipedia 16 .”
MethodVeale, M., Van Kleek, M., & Binns, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174014 · Cited by 490 · cited 1× · see citing works
“…rigor of these approaches, the results of an interview study with 27 public sector machine learning practitioners across 5 OECD countries [73] suggest a disconnect between the current discrimination-aware machine learning research and organizational and institutional realities,…”
BackgroundDigital Discrimination: The Case of Airbnb.com
Edelman, B. G., & Luca, M. (2014) · doi:10.2139/ssrn.2377353 · Cited by 471 · see citing works
Classifying without discriminating
Kamiran, F., & Calders, T. (2009) · 2009 2nd International Conference on Computer, Control and Communication · doi:10.1109/ic4.2009.4909197 · Cited by 409 · see citing works
Decentralization in wikipedia governance.
Forte, A., Larco, V., & Bruckman, A. (2009) · Journal of Management Information Systems · doi:10.2753/mis0742-1222260103 · Cited by 300 · see citing works
Discrimination- and privacy-aware patterns
Hajian, S., Domingo-Ferrer, J., Monreale, A., Pedreschi, D., & Giannotti, F. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0393-7 · Cited by 93 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594 · see citing works
Expanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538 · see citing works
Madaio, M. A., Stark, L., Wortman Vaughan, J., & Wallach, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376445 · Cited by 436 · see citing works
Köbis, N., & Mossink, L. D. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106553 · Cited by 406 · see citing works
Investigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219 · see citing works
Wang, R., Harper, F. M., & Zhu, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376813 · Cited by 211 · see citing works
Self-citationWang, D., Wang, L., Zhang, Z., Wang, D., Zhu, H., Gao, Y., Fan, X., & Tian, F. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445432 · Cited by 209 · see citing works
Self-citationHolstein, K., McLaren, B. M., & Aleven, V. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-23204-7_14 · Cited by 189 · see citing works
Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions
Alkhatib, A., & Bernstein, M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300760 · Cited by 188 · see citing works
Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers
Solanki, P., Grundy, J., & Hussain, W. (2023) · AI and Ethics · doi:10.1007/s43681-022-00195-z · Cited by 157 · see citing works
“To ensure needs are being met, stakeholders should be consulted extensively throughout the AI lifecycle, and particularly during deployment [171].”
BackgroundKawakami, A., Sivaraman, V., Cheng, H.-F., Stapleton, L., Cheng, Y., Qing, D., Perer, A., Wu, Z. S., Zhu, H., & Holstein, K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517439 · Cited by 143 · see citing works
Self-citationSaxena, D., Badillo-Urquiola, K., Wisniewski, P. J., & Guha, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476089 · Cited by 134 · see citing works
This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
Mulligan, D. K., Kroll, J. A., Kohli, N., & Wong, R. Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359221 · Cited by 111 · see citing works
ORES: Lowering Barriers with Participatory Machine Learning in Wikipedia
Halfaker, A., & Geiger, R. S. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415219 · Cited by 82 · see citing works
Self-citationEffect of Information Presentation on Fairness Perceptions of Machine Learning Predictors
van Berkel, N., Goncalves, J., Russo, D., Hosio, S., & Skov, M. B. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445365 · Cited by 84 · see citing works
Smith, C. E., Yu, B., Srivastava, A., Halfaker, A., Terveen, L., & Zhu, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376783 · Cited by 77 · see citing works
Self-citationMetaWriter: Exploring the Potential and Perils of AI Writing Support in Scientific Peer Review
Sun, L., Tao, S., Hu, J., & Dow, S. P. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637371 · Cited by 37 · see citing works
Embedding Societal Values into Social Media Algorithms
Bernstein, M., Christin, A., Hancock, J., Hashimoto, T., Jia, C., Lam, M., Meister, N., Persily, N., Piccardi, T., Saveski, M., et al. (2023) · Journal of Online Trust and Safety · doi:10.54501/jots.v2i1.148 · Cited by 31 · see citing works
Holstein, K., & Doroudi, S. (2021) · arXiv · doi:10.48550/arxiv.2104.12920 · Cited by 29 · see citing works
Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems
Kroll, J. A. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445937 · Cited by 2 · see citing works
- Authors
- Zhu, H., Yu, B., Halfaker, A., & Terveen, L.
- Affiliations
- University of Minnesota (US)
- Wikimedia Foundation (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2018-11
- Volume
- 2
- Issue
- CSCW
- Pages
- 1-23
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W2898911770
- Cited by
- 220
- References
- 10
- Citations by year
- 2026: 16 · 2025: 61 · 2024: 26 · 2023: 31 · 2022: 34 · 2021: 23 · 2020: 16 · 2019: 11 · 2018: 2
- FWCI
- 24.23
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- algorithm design, stakeholder engagement, tacit knowledge, user feedback, algorithmic bias, critical decision-making, human values in AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
