Algorithmic fairness in business analytics: Directions for research and practice
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
The extensive adoption of business analytics (BA) has brought financial gains and increased efficiencies. However, these advances have simultaneously drawn attention to rising legal and ethical challenges when BA inform decisions with fairness implications. As a response to these concerns, the emerging study of algorithmic fairness deals with algorithmic outputs that may result in disparate outcomes or other forms of injustices for subgroups of the population, especially those who have been historically marginalized. Fairness is relevant on the basis of legal compliance, social responsibility, and utility; if not adequately and systematically addressed, unfair BA systems may lead to societal harms and may also threaten an organization's own survival, its competitiveness, and overall performance. This paper offers a forward‐looking, BA‐focused review of algorithmic fairness. We first review the state‐of‐the‐art research on sources and measures of bias, as well as bias mitigation algorithms. We then provide a detailed discussion of the utility–fairness relationship, emphasizing that the frequent assumption of a trade‐off between these two constructs is often mistaken or short‐sighted. Finally, we chart a path forward by identifying opportunities for business scholars to address impactful, open challenges that are key to the effective and responsible deployment of BA.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 13×
“For instance, consider the predictive model meant to estimate health needs to target patients for high-risk care management (Obermeyer et al. 2019).”
BackgroundResultInfluentialDwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287 · cited 1×
“For instance, the potential use of automated radiological screening has recently made waves in the healthcare industry (Choy et al., 2018; McKinney et al., 2020).”
BackgroundBarocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 5×
“To date, BA practices in selected domains have been regulated in the United States, as well as in other countries (cf. Barocas & Selbst, 2016; Kleinberg, Ludwig, Mullainathan, & Sunstein, 2018).”
BackgroundInfluentialEquality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 6×
“These works focus on understanding the different sources of bias, and on effective data-driven techniques for both detecting and mitigating algorithmic bias (e. g., Chouldechova, 2017; Corbett-Davies et al., 2017; Dwork et al., 2012; Hardt & Price, 2016).”
BackgroundMethodInfluentialFair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 7×
“These works focus on understanding the different sources of bias, and on effective data-driven techniques for both detecting and mitigating algorithmic bias (e. g., Chouldechova, 2017; Corbett-Davies et al., 2017; Dwork et al., 2012; Hardt & Price, 2016).”
BackgroundInfluentialMitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387 · cited 2×
“Specifically, one can formulate it as a minimax game that can then be estimated via adversarial learning (Zhang et al., 2018).”
BackgroundData preprocessing techniques for classification without discrimination
Kamiran, F., & Calders, T. (2012) · Knowledge and Information Systems · doi:10.1007/s10115-011-0463-8 · Cited by 1,305 · cited 1×
“This can be done by reweighting, where the frequency of observations from the minority group is artificially increased through resampling and thus the observations from both disadvantaged and advantaged group become more balanced (Kamiran & Calders, 2012).”
BackgroundDietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230
Amazon Scraps Secret AI Recruiting Tool that Showed Bias against Women *
Dastin, J. (2022) · Ethics of Data and Analytics · doi:10.1201/9781003278290-44 · Cited by 1,165
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
The accuracy, fairness, and limits of predicting recidivism
Dressel, J., & Farid, H. (2018) · Science Advances · doi:10.1126/sciadv.aao5580 · Cited by 1,071 · cited 1×
“Research has shown that assessments collected via crowdsourcing platforms are sensitive to societal biases (Dressel & Farid, 2018).”
BackgroundHuman Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
Big Data Analytics in Operations Management
Choi, T., Wallace, S. W., & Wang, Y. (2018) · Production and Operations Management · doi:10.1111/poms.12838 · Cited by 1,027
Inherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
Lambrecht, A., & Tucker, C. (2019) · Management Science · doi:10.1287/mnsc.2018.3093 · Cited by 854
Mitigating bias in algorithmic hiring: evaluating claims and practices
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372828 · Cited by 845
Current Applications and Future Impact of Machine Learning in Radiology
Choy, G., Khalilzadeh, O., Michalski, M., Do, S., Samir, A. E., Pianykh, O. S., Geis, J. R., Pandharipande, P. V., Brink, J. A., & Dreyer, K. J. (2018) · Radiology · doi:10.1148/radiol.2018171820 · Cited by 800 · cited 1×
“For instance, the potential use of automated radiological screening has recently made waves in the healthcare industry (Choy et al., 2018; McKinney et al., 2020).”
BackgroundImproving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · cited 1×
“In particular, interviews with analytics practitioners (Holstein et al., 2019) suggest that, perhaps not unexpectedly, more support is needed to guide practitioners in auditing models.”
BackgroundFairness-Aware Classifier with Prejudice Remover Regularizer
Kamishima, T., Akaho, S., Asoh, H., & Sakuma, J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33486-3_3 · Cited by 694 · cited 1×
“In-processing incorporates the fairness goal directly in the modeling objective (Kamishima et al., 2012; Wang & Saar-Tsechansky, 2020; Zafar et al., 2019).”
MethodPredictive Policing: The Role of Crime Forecasting in Law Enforcement Operations
Perry, W., McInnis, B., Price, C., Smith, S., & Hollywood, J. (2013) · doi:10.7249/rr233 · Cited by 693
Algorithmic Fairness: Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042720-125902 · Cited by 687 · cited 4×
“…et al., 2017), and existing surveys provide perspectives from computer science (Chouldechova & Roth, 2020; Corbett-Davies & Goel, 2018), statistics (Mitchell et al., 2021), philosophy (Danks & London, 2017; Fazelpour & Danks, 2021), law (Barocas & Selbst, 2016), and economics (Cowgill & Tucker,…”
BackgroundInfluentialPrediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2018) · arXiv · doi:10.48550/arxiv.1811.07867 · Cited by 687
Raji, I. D., & Buolamwini, J. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314244 · Cited by 651
Machine Learning and Health Care Disparities in Dermatology
Adamson, A. S., & Smith, A. (2018) · JAMA Dermatology · doi:10.1001/jamadermatol.2018.2348 · Cited by 639 · cited 2×
“Consider again the example of presentation of dermatological symptoms across different skin colors (Adamson & Smith, 2018).”
BackgroundLum, K., & Isaac, W. (2016) · Significance · doi:10.1111/j.1740-9713.2016.00960.x · Cited by 611
Algorithmic Bias in Autonomous Systems
Danks, D., & London, A. J. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/654 · Cited by 472 · cited 1×
“…al., 2017), and existing surveys provide perspectives from computer science (Chouldechova & Roth, 2020; Corbett-Davies & Goel, 2018), statistics (Mitchell et al., 2021), philosophy (Danks & London, 2017; Fazelpour & Danks, 2021), law (Barocas & Selbst, 2016), and economics (Cowgill & Tucker, 2020).”
The Measure and Mismeasure of Fairness
Corbett-Davies, S., Gaebler, J. D., Nilforoshan, H., Shroff, R., & Goel, S. (2018) · arXiv · doi:10.48550/arxiv.1808.00023 · Cited by 444
Discrimination in the Age of Algorithms
Kleinberg, J., Ludwig, J., Mullainathan, S., & Sunstein, C. R. (2018) · Journal of Legal Analysis · doi:10.1093/jla/laz001 · Cited by 401
Underspecification Presents Challenges for Credibility in Modern Machine Learning
D'Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M. D., et al. (2020) · arXiv · doi:10.48550/arxiv.2011.03395 · Cited by 398 · cited 1×
“In the context of predictive modeling, this phenomenon has been studied as underspecification (D’Amour, Heller, et al., 2020) or predictive multiplicity (Marx et al., 2020).”
MethodBias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
De-Arteaga, M., Romanov, A., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S., Kenthapadi, K., & Kalai, A. T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287572 · Cited by 337 · cited 5×
“Meanwhile, in a task such as predicting someone’s occupation to show them relevant job postings, the allocation of a good (e. g., showing a relevant job) may justify a focus on differential false negative rates (De-Arteaga et al., 2019).”
BackgroundInfluentialSelf-citationBias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
De-Arteaga, M., Romanov, A., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S., Kenthapadi, K., & Kalai, A. T. (2019) · arXiv · doi:10.48550/arxiv.1901.09451 · Cited by 337
Self-citationAccountability and automation bias
SKITKA, L. J., MOSIER, K., & BURDICK, M. D. (2000) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1999.0349 · Cited by 324 · cited 1×
“Broadly speaking, decision makers’ likelihood to adhere to algorithmic recommendations relates to phenomena of algorithm aversion (Dietvorst et al., 2018) and automation bias (Skitka et al., 2000).”
BackgroundSenoner, J., Netland, T., & Feuerriegel, S. (2022) · Management Science · doi:10.1287/mnsc.2021.4190 · Cited by 317
Self-citationKleinberg, J., Ludwig, J., Mullainathan, S., & Rambachan, A. (2018) · AEA Papers and Proceedings · doi:10.1257/pandp.20181018 · Cited by 278
Mitigating bias in machine learning for medicine
Vokinger, K. N., Feuerriegel, S., & Kesselheim, A. S. (2021) · Communications Medicine · doi:10.1038/s43856-021-00028-w · Cited by 269
Self-citationFairness in Learning: Classic and Contextual Bandits
Joseph, M., Kearns, M., Morgenstern, J., & Roth, A. (2016) · arXiv · doi:10.48550/arxiv.1605.07139 · Cited by 199 · cited 2×
“For instance, fairness constraints have been incorporated into a classic bandit (Joseph et al. 2016).”
BackgroundFairness in credit scoring: Assessment, implementation and profit implications
Kozodoi, N., Jacob, J., & Lessmann, S. (2022) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.06.023 · Cited by 198
Runaway Feedback Loops in Predictive Policing
Ensign, D., Friedler, S. A., Neville, S., Scheidegger, C., & Venkatasubramanian, S. (2017) · arXiv · doi:10.48550/arxiv.1706.09847 · Cited by 161 · cited 1×
“Researchers have showed that relying on policing data to induce models perpetuates racial biases in policing practices and leads to feedback loops in police allocation (Ensign et al., 2018; Lum & Isaac, 2016).”
BackgroundTargeting Prospective Customers: Robustness of Machine-Learning Methods to Typical Data Challenges
Simester, D., Timoshenko, A., & Zoumpoulis, S. I. (2020) · Management Science · doi:10.1287/mnsc.2019.3308 · Cited by 133
Inherent Trade-Offs in Algorithmic Fairness
Kleinberg, J. (2018) · Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems · doi:10.1145/3219617.3219634 · Cited by 125
What's sex got to do with machine learning?
Hu, L., & Kohler-Hausmann, I. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375674 · Cited by 74
Impact of risk assessment on judges’ fairness in sentencing relatively poor defendants
Skeem, J., Scurich, N., & Monahan, J. (2020) · Law and Human Behavior · doi:10.1037/lhb0000360 · Cited by 70 · cited 2×
“In the criminal justice system, differential adherence has been shown to exacerbate disparities in incarceration across Black and White defendants (Albright, 2019), and across socioeconomic brackets (Skeem et al., 2020).”
BackgroundRodolfa, K. T., Lamba, H., & Ghani, R. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00396-x · Cited by 10 · cited 1×
“Recent work showing that fairness-accuracy trade-offs can be negligible (Rodolfa et al., 2021) highlights the importance of empirical evidence to guide our understanding of the cost/benefit relationship of fairness-oriented efforts.”
Background
Cited by
Feuerriegel, S., Hartmann, J., Janiesch, C., & Zschech, P. (2024) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00834-7 · Cited by 2,267
Self-citationFrom Scarcity to Abundance: Scholars and Scholarship in an Age of Generative Artificial Intelligence
Grimes, M., von Krogh, G., Feuerriegel, S., Rink, F., & Gruber, M. (2023) · Academy of Management Journal · doi:10.5465/amj.2023.4006 · Cited by 180
Self-citationNyberg, A. J., Schleicher, D. J., Bell, B. S., Boon, C., Cappelli, P., Collings, D. G., Dalle Molle, J. E., Feuerriegel, S., Gerhart, B., Jeong, Y., et al. (2025) · Journal of Management · doi:10.1177/01492063251325188 · Cited by 47
Self-citationMoorthy, U. M. K., Muthukumaran, A. M. J., Kaliyaperumal, V., Jayakumar, S., & Vijayaraghavan, K. A. (2025) · Explainable Artificial Intelligence in the Healthcare Industry · doi:10.1002/9781394249312.ch23 · Cited by 8
Metadata
- Authors
- De‐Arteaga, M., Feuerriegel, S., & Saar‐Tsechansky, M.
- Venue
- Production and Operations Management
- Publisher
- SAGE Publications
- Published
- 2022-10
- Volume
- 31
- Issue
- 10
- Pages
- 3749-3770
- ISSN
- 1059-1478 (ISSN-L)
- 1937-5956
- OpenAlex
- W4292574939
- Cited by
- 87
- References
- 44
- FWCI
- 19.62
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.