A sociotechnical view of algorithmic fairness
Open accessThe DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
Abstract
Algorithmic fairness (AF) has been framed as a newly emerging technology that mitigates systemic discrimination in automated decision‐making, providing opportunities to improve fairness in information systems (IS). However, based on a state‐of‐the‐art literature review, we argue that fairness is an inherently social concept and that technologies for AF should therefore be approached through a sociotechnical lens. We advance the discourse on AF as a sociotechnical phenomenon. Our research objective is to embed AF in the sociotechnical view of IS. Specifically, we elaborate on why outcomes of a system that uses algorithmic means to assure fairness depend on mutual influences between technical and social structures. This perspective can generate new insights that integrate knowledge from both technical fields and social studies. Further, it spurs new directions for IS debates. We contribute as follows: First, we problematize fundamental assumptions in the current discourse on AF based on a systematic analysis of 310 articles. Second, we respond to these assumptions by theorizing AF as a sociotechnical construct. Third, we propose directions for IS researchers to enhance their impacts by pursuing a unique understanding of sociotechnical AF. We call for and undertake a holistic approach to AF. A sociotechnical perspective on AF can yield holistic solutions to systemic biases and discrimination.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“A widely used algorithm for selecting healthcare system entry was found to discriminate against racial minorities, thereby affecting thousands of people (Obermeyer et al., 2019).”
MethodTowards an understanding of inequity
Adams, J. S. (1963) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0040968 · Cited by 5,146 · cited 1×
“Finally, psychology sees fairness as a perception of an individual comparing themselves to other individuals whom they consider to be relational partners (ie, as somehow similar to them) (Adams, 1963).”
BackgroundDwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 1×
“Other group-level notions of fairness rely on prediction errors (Hardt & Price, 2016; Kleinberg, Mullainathan, & Raghavan, 2017; Zafar, Valera, Rodriguez, Gummadi, & Weller, 2017), whereby the confusion matrix from Figure 1 is considered.”
BackgroundMitchell, M. (2009) · doi:10.1093/oso/9780195124415.001.0001 · Cited by 2,310
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 1×
“As can be seen in the different notions, there is no universal operationalization of fairness; further, it is mathematically impossible to fulfil all the different notions of fairness at once (Chouldechova, 2017; Kleinberg et al., 2017).”
BackgroundThe Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907
Where the Action Is: The Foundations of Embodied Interaction
Dourish, P. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/7221.001.0001 · Cited by 1,866
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631
Ensuring fairness in machine learning to advance health equity.
Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G., & Chin, M. H. (2018) · Annals of Internal Medicine · doi:10.7326/m18-1990 · Cited by 1,341
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Fairness definitions explained
Verma, S., & Rubin, J. (2018) · Proceedings of the International Workshop on Software Fairness · doi:10.1145/3194770.3194776 · Cited by 1,124
Bias in data‐driven artificial intelligence systems—An introductory survey
Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., et al. (2020) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1356 · Cited by 1,043
In AI we trust? Perceptions about automated decision-making by artificial intelligence
Araujo, T., Helberger, N., Kruikemeier, S., & de Vreese, C. H. (2020) · AI & SOCIETY · doi:10.1007/s00146-019-00931-w · Cited by 1,038
Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995 · cited 4×
“The counterfactual fairness discourse has pursued a view of AF that uses directed acyclic graphs to model social biases that may occur in data (Coston et al., 2020; Kusner et al., 2017).”
BackgroundInfluentialInherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
Moralizing Technology: Understanding and Designing the Morality of Things
Verbeek, P.-P. (2011) · doi:10.7208/chicago/9780226852904.001.0001 · Cited by 923
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence
Mohamed, S., Png, M.-T., & Isaac, W. (2020) · Philosophy & Technology · doi:10.1007/s13347-020-00405-8 · Cited by 912 · cited 5×
“ences (Hoffmann, 2019), philosophy (Mohamed et al., 2020; Wong, 2020), and criminology and law (Barocas & Selbst, 2016; Helberger, Araujo, & de Vreese, 2020; Završnik, 2019).”
BackgroundMethodInfluentialMachines as teammates: A research agenda on AI in team collaboration
Seeber, I., Bittner, E., Briggs, R. O., de Vreede, T., de Vreede, G.-J., Elkins, A., Maier, R., Merz, A. B., Oeste-Reiß, S., Randrup, N., et al. (2020) · Information & Management · doi:10.1016/j.im.2019.103174 · Cited by 835
Self-citationAlgorithmic bias: review, synthesis, and future research directions
Kordzadeh, N., & Ghasemaghaei, M. (2022) · European Journal of Information Systems · doi:10.1080/0960085x.2021.1927212 · Cited by 770
Lepri, B., Oliver, N., Letouzé, E., Pentland, A., & Vinck, P. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0279-x · Cited by 709
Value Sensitive Design and Information Systems
Friedman, B., Kahn, P. H., Borning, A., & Huldtgren, A. (2013) · Philosophy of Engineering and Technology · doi:10.1007/978-94-007-7844-3_4 · Cited by 678 · cited 2×
“Similarly, IS has addressed technology development as a sociotechnical endeavour (Hassan & Mathiassen, 2018) in which the values of involved individuals become embedded in the created artefacts (Chatterjee, Sarker, & Fuller, 2009; Friedman et al., 2013; van de Poel & Kroes, 2014).”
BackgroundFairness in Machine Learning: Lessons from Political Philosophy
Binns, R. (2017) · arXiv · doi:10.48550/arxiv.1712.03586 · Cited by 676
Kuziemski, M., & Misuraca, G. (2020) · Telecommunications Policy · doi:10.1016/j.telpol.2020.101976 · Cited by 610
A comparative study of fairness-enhancing interventions in machine learning
Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., & Roth, D. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287589 · Cited by 601
General System Theory: Applications for Organization and Management.
Kast, F. E., & Rosenzweig, J. E. (1972) · Academy of Management Journal · doi:10.2307/255141 · Cited by 599
Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse
Hoffmann, A. L. (2019) · Information, Communication & Society · doi:10.1080/1369118x.2019.1573912 · Cited by 535 · cited 5×
“…2020; Binns, 2018; Mulligan et al., 2019; Robert et al., 2020) as well as in outlets from other disciplines, including the social sciences (Hoffmann, 2019), philosophy (Mohamed et al., 2020; Wong, 2020), and criminology and law (Barocas & Selbst, 2016; Helberger, Araujo, & de Vreese,…”
BackgroundInfluentialVeale, M., & Binns, R. (2017) · doi:10.31235/osf.io/ustxg · Cited by 512
Sejnowski, T. J. (2018) · doi:10.7551/mitpress/11474.001.0001 · Cited by 438
Identifying Ethical Considerations for Machine Learning Healthcare Applications
Char, D. S., Abràmoff, M. D., & Feudtner, C. (2020) · The American Journal of Bioethics · doi:10.1080/15265161.2020.1819469 · Cited by 415
Artificial intelligence vs COVID-19: limitations, constraints and pitfalls
Naudé, W. (2020) · AI & SOCIETY · doi:10.1007/s00146-020-00978-0 · Cited by 368
Ai for social good: unlocking the opportunity for positive impact
Tomašev, N., Cornebise, J., Hutter, F., Mohamed, S., Picciariello, A., Connelly, B., Belgrave, D. C. M., Ezer, D., Haert, F. C. V. D., Mugisha, F., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-15871-z · Cited by 357
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
De-Arteaga, M., Romanov, A., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S., Kenthapadi, K., & Kalai, A. T. (2019) · arXiv · doi:10.48550/arxiv.1901.09451 · Cited by 337
Emerging models of data governance in the age of datafication
Micheli, M., Ponti, M., Craglia, M., & Berti Suman, A. (2020) · Big Data & Society · doi:10.1177/2053951720948087 · Cited by 332
Lysaght, T., Lim, H. Y., Xafis, V., & Ngiam, K. Y. (2019) · Asian Bioethics Review · doi:10.1007/s41649-019-00096-0 · Cited by 332
Senoner, J., Netland, T., & Feuerriegel, S. (2022) · Management Science · doi:10.1287/mnsc.2021.4190 · Cited by 317
Self-citationHacker, P. (2018) · Common Market Law Review · doi:10.54648/cola2018095 · Cited by 317
Towards a critical race methodology in algorithmic fairness
Hanna, A., Denton, R., Smart, A., & Smith-Loud, J. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372826 · Cited by 295 · cited 1×
“does not work properly for them, this impacts on their dignity and belongs to the interactional justice dimension (Celis Bueno, 2020; Hanna et al., 2020; Robert et al., 2020).”
BackgroundBlockchain technology as a regulatory technology: from code is law to law is code
Filippi, P. D., & Hassan, S. (2016) · First Monday · doi:10.5210/fm.v21i12.7113 · Cited by 284
Measuring discrimination in algorithmic decision making
Žliobaitė, I. (2017) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-017-0506-1 · Cited by 272
Mitigating bias in machine learning for medicine
Vokinger, K. N., Feuerriegel, S., & Kesselheim, A. S. (2021) · Communications Medicine · doi:10.1038/s43856-021-00028-w · Cited by 269
Self-citationDesigning fair AI for managing employees in organizations: a review, critique, and design agenda
Robert, L. P., Pierce, C., Marquis, L., Kim, S., & Alahmad, R. (2020) · Human–Computer Interaction · doi:10.1080/07370024.2020.1735391 · Cited by 268
A Qualitative Exploration of Perceptions of Algorithmic Fairness
Woodruff, A., Fox, S. E., Rousso-Schindler, S., & Warshaw, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174230 · Cited by 256
Yang, K., Qinami, K., Fei-Fei, L., Deng, J., & Russakovsky, O. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375709 · Cited by 254
Lessons from archives: strategies for collecting sociocultural data in machine learning
Jo, E. S., & Gebru, T. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372829 · Cited by 240
How computer systems embody values
Nissenbaum, H. (2001) · Computer · doi:10.1109/2.910905 · Cited by 232
Artificial intelligence as digital agency
Ågerfalk, P. J. (2020) · European Journal of Information Systems · doi:10.1080/0960085x.2020.1721947 · Cited by 217 · cited 6×
“While the discourse on explainable AI has reached IS (Ågerfalk, 2020; B. Kim, Park, & Suh, 2020), it remains nascent.”
BackgroundInfluentialBrown, A., Chouldechova, A., Putnam-Hornstein, E., Tobin, A., & Vaithianathan, R. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300271 · Cited by 207
Decision Support Systems: A Historical Overview
Power, D. J. (2008) · International Handbooks on Information Systems · doi:10.1007/978-3-540-48713-5_7 · Cited by 202
Scheuerman, M. K., Wade, K., Lustig, C., & Brubaker, J. R. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392866 · Cited by 193
Democratizing Algorithmic Fairness
Wong, P.-H. (2020) · Philosophy & Technology · doi:10.1007/s13347-019-00355-w · Cited by 182
Kim, B., Park, J., & Suh, J. (2020) · Decision Support Systems · doi:10.1016/j.dss.2020.113302 · Cited by 182
Templier, M., & Paré, G. (2018) · European Journal of Information Systems · doi:10.1080/0960085x.2017.1398880 · Cited by 180
Srivastava, M., Heidari, H., & Krause, A. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330664 · Cited by 179 · cited 2×
“This contradicts the evidence that proved that fairness assessments are highly context-dependent (Srivastava et al., 2019b; Wong, 2020; Zafar et al., 2017).”
BackgroundJohndrow, J. E., & Lum, K. (2019) · The Annals of Applied Statistics · doi:10.1214/18-aoas1201 · Cited by 141
Abràmoff, M. D., Tobey, D., & Char, D. S. (2020) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2020.02.022 · Cited by 140
Algorithmic Decision-Making in AVs: Understanding Ethical and Technical Concerns for Smart Cities
Lim, H. S. M., & Taeihagh, A. (2019) · Sustainability · doi:10.3390/su11205791 · Cited by 136
Inherent Trade-Offs in Algorithmic Fairness
Kleinberg, J. (2018) · Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems · doi:10.1145/3219617.3219634 · Cited by 125
Algorithmic equity in the hiring of underrepresented IT job candidates
Yarger, L., Cobb Payton, F., & Neupane, B. (2019) · Online Information Review · doi:10.1108/oir-10-2018-0334 · Cited by 122 · cited 1×
“, 2020), social order (Lepri et al., 2018; Mohamed et al., 2020; Rosenberger, 2020; Wellner & Rothman, 2020; Yarger et al., 2019), or public discourse around AI (Araujo et al.”
BackgroundThis Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
Mulligan, D. K., Kroll, J. A., Kohli, N., & Wong, R. Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359221 · Cited by 111
Dwork, C., & Ilvento, C. (2019) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2019.33 · Cited by 91 · cited 2×
“…(Chouldechova, Benavides-Prado, Fialko, & Vaithianathan, 2018; Kallus & Zhou, 2019; Rahmattalabi et al., 2019), accepted temporal dynamics of fairness issues (D'Amour et al., 2020; Liu, Dean, Rolf, Simchowitz, & Hardt, 2018), or addressed technical systems in a holistic way (Dwork & Ilvento, 2018).”
BackgroundPredict Responsibly: Improving Fairness and Accuracy by Learning to Defer
Madras, D., Pitassi, T., & Zemel, R. (2017) · arXiv · doi:10.48550/arxiv.1711.06664 · Cited by 90
Residual unfairness in fair machine learning from prejudiced data
Kallus, N., & Zhou, A. (2018) · arXiv · doi:10.48550/arxiv.1806.02887 · Cited by 83
What's sex got to do with machine learning?
Hu, L., & Kohler-Hausmann, I. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375674 · Cited by 74
Fair Algorithms for Learning in Allocation Problems
Elzayn, H., Jabbari, S., Jung, C., Kearns, M., Neel, S., Roth, A., & Schutzman, Z. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287571 · Cited by 70
Differentially Private Fair Learning
Jagielski, M., Kearns, M., Mao, J., Oprea, A., Roth, A., Sharifi-Malvajerdi, S., & Ullman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.02696 · Cited by 62
Fairwashing: the risk of rationalization
Aïvodji, U., Arai, H., Fortineau, O., Gambs, S., Hara, S., & Tapp, A. (2019) · arXiv · doi:10.48550/arxiv.1901.09749 · Cited by 54
Glymour, B., & Herington, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287573 · Cited by 50
Fair Regression with Wasserstein Barycenters
Chzhen, E., Denis, C., Hebiri, M., Oneto, L., & Pontil, M. (2020) · arXiv · doi:10.48550/arxiv.2006.07286 · Cited by 46
(2020) · doi:10.1145/3351095.3375784 · Cited by 29
When the System Does Not Fit: Coping Strategies of Employment Consultants
Dolata, M., Schenk, B., Fuhrer, J., Marti, A., & Schwabe, G. (2020) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-020-09377-x · Cited by 22
Self-citation
Cited by
Feuerriegel, S., Hartmann, J., Janiesch, C., & Zschech, P. (2024) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00834-7 · Cited by 2,267
“Addressing bias and thus fairness in AI receives increasing attention in the academic literature (DeArteaga et al., 2022; Dolata et al., 2022; Ferrara, 2023; Feuerriegel et al., 2020; Schramowski et al., 2022; von Zahn et al., 2022), but remains an open and ongoing research question.”
Self-citationDigital governance: A conceptual framework and research agenda
Hanisch, M., Goldsby, C. M., Fabian, N. E., & Oehmichen, J. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2023.113777 · Cited by 267
AI Knowledge: Improving AI Delegation through Human Enablement
Pinski, M., Adam, M., & Benlian, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580794 · Cited by 62
“In addition to human preference, researchers further agree that human-AI collaboration must match ethical standards [31].”
BackgroundMobilising new frontiers in digital transformation research: A problematization review
Ashrafi, A., Constantinides, P., Mehandjiev, N., & Thatcher, J. B. (2025) · Information Systems Journal · doi:10.1111/isj.12531 · Cited by 33
“Problematization reviews, on the other hand, center on the critical examination of the underlying theoretical and conceptual foundations in a specific domain of study (Dolata et al., 2022).”
A multi-perspective framework for research on (sustainable) autonomous systems
Beck, R., Dibbern, J., & Wiener, M. (2022) · Business & Information Systems Engineering · doi:10.1007/s12599-022-00752-0 · Cited by 23
“Here, critical ethical questions arise (Tang et al. 2020), including questions of fairness regarding the decision rules according to which AS act (Dolata et al. 2021); for instance, how a driverless car should react to unforeseen circumstances affecting humans (Kirkpatrick 2015).”
Background
Metadata
- Authors
- Dolata, M., Feuerriegel, S., & Schwabe, G.
- Venue
- Information Systems Journal
- Publisher
- Wiley
- Published
- 2021-10-07
- Volume
- 32
- Issue
- 4
- Pages
- 754-818
- ISSN
- 1350-1917 (ISSN-L)
- 1365-2575
- OpenAlex
- W3203106571
- Cited by
- 160
- References
- 73
- FWCI
- 25.64
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.