The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Algorithmic bias: Senses, sources, solutions
Open accessAbstract
Data‐driven algorithms are widely used to make or assist decisions in sensitive domains, including healthcare, social services, education, hiring, and criminal justice. In various cases, such algorithms have preserved or even exacerbated biases against vulnerable communities, sparking a vibrant field of research focused on so‐called algorithmic biases. This research includes work on identification, diagnosis, and response to biases in algorithm‐based decision‐making. This paper aims to facilitate the application of philosophical analysis to these contested issues by providing an overview of three key topics: What is algorithmic bias? Why and how can it occur? What can and should be done about it? Throughout, we highlight connections—both actual and potential—with philosophical ideas and concerns.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“In particular, there can be disparate impacts , where members of a protected group are differentially impacted relative to a (more) dominant group (Obermeyer et al., 2019).”
BackgroundAlgorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
Algorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631
Data preprocessing techniques for classification without discrimination
Kamiran, F., & Calders, T. (2012) · Knowledge and Information Systems · doi:10.1007/s10115-011-0463-8 · Cited by 1,305
Fairness in Criminal Justice Risk Assessments: The State of the Art
Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021) · Sociological Methods & Research · doi:10.1177/0049124118782533 · Cited by 988 · cited 1×
“…different ones are connected to concepts 7 such as: equality of treatment (e.g., Dieterich et al., 2016), the legal doctrines of disparate treatment (e.g., Berk et al., 2021) and disparate impact (e.g., Feldman et al., 2015), and the ideal of equality of opportunity (e.g., Hardt et al., 2016).”
BackgroundInherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · cited 2×
“…strategies can lead to significant social costs depending on the degree to which fairness gains are ‘traded off’ for epistemically valuable measures (e.g., predictive accuracy) that also matter for societal welfare (Corbett ‐ Davies, Pierson, et al., 2017; Gendler, 2011; Kleinberg et al., 2016).”
BackgroundResultAlgorithmic Fairness: Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042720-125902 · Cited by 687
A large-scale analysis of racial disparities in police stops across the United States
Pierson, E., Simoiu, C., Overgoor, J., Corbett-Davies, S., Jenson, D., Shoemaker, A., Ramachandran, V., Barghouty, P., Phillips, C., Shroff, R., et al. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0858-1 · Cited by 552 · cited 1×
“In fact, there are even recent efforts to use the presence of algorithmic bias as a ‘signal’ of previously unnoticed real ‐ world biases that should potentially be ameliorated (Carton et al., 2016; Pierson et al., 2020).”
BackgroundWhere fairness fails: data, algorithms, and the limits of antidiscrimination discourse
Hoffmann, A. L. (2019) · Information, Communication & Society · doi:10.1080/1369118x.2019.1573912 · Cited by 535
Algorithmic Bias in Autonomous Systems
Danks, D., & London, A. J. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/654 · Cited by 472 · cited 3×
“At its most neutral, algorithmic bias is simply systematic deviation in algorithm output, performance, or impact, relative to some norm or standard (Ant-ony, 2016; Danks & London, 2017; Johnson, 2020).”
BackgroundMethodSelf-citationDiscrimination in the Age of Algorithms
Kleinberg, J., Ludwig, J., Mullainathan, S., & Sunstein, C. R. (2018) · Journal of Legal Analysis · doi:10.1093/jla/laz001 · Cited by 401
50 Years of Test (Un)fairness: Lessons for Machine Learning
Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287600 · Cited by 297
Problem Formulation and Fairness
Passi, S., & Barocas, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287567 · Cited by 283
Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments
Green, B., & Chen, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287563 · Cited by 275
A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
De-Arteaga, M., Fogliato, R., & Chouldechova, A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376638 · Cited by 221
The Selective Labels Problem: Evaluating Algorithmic Predictions in the Presence of Unobservables
Lakkaraju, H., Kleinberg, J., Leskovec, J., Ludwig, J., & Mullainathan, S. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098066 · Cited by 138
Why Yellow Fever Isn't Flattering: A Case Against Racial Fetishes
ZHENG, R. (2016) · Journal of the American Philosophical Association · doi:10.1017/apa.2016.25 · Cited by 125 · cited 1×
“Philosophical by Zheng 2016) in contexts such as image classification and natural language processing (see also Eubanks, 2018 on algorithm ‐ induced stigmatization and stereotyping).”
BackgroundIdentifying Police Officers at Risk of Adverse Events
Carton, S., Helsby, J., Joseph, K., Mahmud, A., Park, Y., Walsh, J., Cody, C., Patterson, C. E., Haynes, L., & Ghani, R. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939698 · Cited by 64
Glymour, B., & Herington, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287573 · Cited by 50
Research works printed but not linked: 67
- 10.1177/2053951720949566 · DOI · Paper page
- 10.1515/9781400836826 · DOI · Paper page
- 10.1093/acprof:oso/9780198713241.003.0007 · DOI
- Austin N. L., 1990, The role of racial identity in Black students' attitudes toward counseling and counseling centers, Journal of College Student Development
- Barabas C., 2018, Conference on Fairness, Accountability and Transparency, 62
- 10.1111/hypa.12170 · DOI · Paper page
- Benjamin R., 2019, Race after technology: Abolitionist tools for the New Jim Code
- 10.1515/pjbr-2019-0004 · DOI · Paper page
- Buolamwini J., 2018, Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification, Journal of Machine Learning Research, 81, 1
- Castro C., 2019, What's wrong with machine bias, Ergo, 6
- 10.1145/3376898 · DOI · Paper page
- Corbett‐Davies S., 2018, The measure and mismeasure of fairness: A critical review of fair machine learning
- Corbett‐Davies S., 2017, Even imperfect algorithms can improve the criminal justice system, New York Times
- 10.1145/3097983.3098095 · DOI · Paper page
- 10.1038/d41586-018-05469-3 · DOI · Paper page
- 10.1126/science.aba9647 · DOI · Paper page
- 10.1086/709729 · DOI · Paper page
- Dai J., 2021, Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society
- 10.1145/3306618.3314228 · DOI
- De‐Arteaga M., 2018, Learning under selective labels in the presence of expert consistency
- 10.1016/j.dss.2010.06.003 · DOI · Paper page
- Dieterich W., 2016, COMPAS risk scales: Demonstrating accuracy equity and predictive parity, Northpoint Inc, 7, 1
- Dotan R., 2020, Theory choice, non‐epistemic values, and machine learning, Synthese, 1
- 10.2307/j.ctt6wrc78 · DOI · Paper page
- 10.1145/2090236.2090255 · DOI · Paper page
- Dwork C., 2020, 1st Symposium on Foundations of Responsible Computing (FORC 2020)
- Ensign D., 2018, Conference on Fairness, Accountability and Transparency, 160
- Eubanks V., 2018, Automating inequality
- 10.1145/3375627.3375828 · DOI
- 10.7551/mitpress/7585.001.0001 · DOI · Paper page
- 10.1007/s11098-011-9801-7 · DOI · Paper page
- Hacking I., 1995, Causal cognition: A multidisciplinary debate
- 10.1093/annonc/mdy166 · DOI · Paper page
- Hart H. L. A., 1961, The concept of law
- Hellman D., 2020, Measuring algorithmic fairness, Virginia Law Review, 106, 811
- 10.1145/3178876.3186044 · DOI
- 10.1145/3351095.3375674 · DOI · Paper page
- Johnson G.(2020).Algorithmic bias: On the implicit biases of social technology.Synthese 1‐21. · DOI · Paper page
- Johnson G.(In press).Are algorithms value‐free? Feminist theoretical virtues in machine learning. Journal of Moral Philosophy.
- Kallus N., 2018, Proceedings of the 35th International Conference on Machine Learning
- Kearns M., 2019, The ethical algorithm: The science of socially aware algorithm design
- Kilbertus N., 2017, Advances in Neural Information Processing Systems, 30, 656
- Kleinberg J., 2018, Human decisions and machine predictions, Quarterly Journal of Economics, 133, 237, 10.1093/qje/qjx032 · DOI · Paper page
- 10.1007/s13347-019-00372-9 · DOI · Paper page
- 10.1038/d41586-020-00274-3 · DOI
- Kusner M. J., 2017, Counterfactual fairness
- 10.1145/3375627.3375808 · DOI
- Liao S., Oppressive things, Philosophy and Phenomenological Research
- Liu L. T., 2018, International Conference on Machine Learning, 3150
- Loftus J. R., 2018, Causal reasoning for algorithmic fairness
- 10.1002/hast.973 · DOI · Paper page
- 10.1037/a0016539 · DOI · Paper page
- Mayson S. G., 2018, Bias in, bias out, Yale Law Journal, 128, 2218
- 10.7208/chicago/9780226261317.001.0001 · DOI · Paper page
- 10.1145/3287560.3287576 · DOI
- 10.4324/9781315545882 · doi:10.4324/9781315545882
- 10.18574/nyu/9781479833641.001.0001 · DOI · Paper page
- O'Neil C., 2016, Weapons of math destruction: How big data increases inequality and threatens democracy
- 10.2307/1122890 · doi:10.2307/1122890
- Rambachan A., 2020, 1st Symposium on Foundations of Responsible Computing (FORC 2020)
- Simon J., 2017, The ethics of technology, 219
- 10.1016/j.eswa.2013.07.046 · DOI · Paper page
- 10.1007/s11229-010-9793-4 · DOI
- Véliz C., 2020, Privacy is power: Why and how you should take back control of your data
- 10.2307/1340593 · DOI · Paper page
- Winner L., 1980, Do artifacts have politics?, Dædalus, 121
- Zafar M. B., 2017, Artificial Intelligence and Statistics, 962
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Six Human-Centered Artificial Intelligence Grand Challenges
Ozmen Garibay, O., Winslow, B., Andolina, S., Antona, M., Bodenschatz, A., Coursaris, C., Falco, G., Fiore, S. M., Garibay, I., Grieman, K., et al. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2153320 · Cited by 501
AI and the transformation of social science research
Grossmann, I., Feinberg, M., Parker, D. C., Christakis, N. A., Tetlock, P. E., & Cunningham, W. A. (2023) · Science · doi:10.1126/science.adi1778 · Cited by 377
Algorithmic Political Bias in Artificial Intelligence Systems
Peters, U. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00512-8 · Cited by 111
Trust, trustworthiness and AI governance
Lahusen, C., Maggetti, M., & Slavkovik, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-71761-0 · Cited by 68
Metadata
- Authors
- Fazelpour, S., & Danks, D.
- Venue
- Philosophy Compass
- Publisher
- Wiley
- Published
- 2021-06-12
- Volume
- 16
- Issue
- 8
- ISSN
- 1747-9991 (ISSN-L)
- OpenAlex
- W3168799536
- Cited by
- 272
- References
- 25
- FWCI
- 46.34
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Funders
- Social Sciences and Humanities Research Council of Canada (#756‐2019‐0289 2)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.