Managing Bias in AI
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Recent awareness of the impacts of bias in AI algorithms raises the risk for companies to deploy such algorithms, especially because the algorithms may not be explainable in the same way that non-AI algorithms are. Even with careful review of the algorithms and data sets, it may not be possible to delete all unwanted bias, particularly because AI systems learn from historical data, which encodes historical biases. In this paper, we propose a set of processes that companies can use to mitigate and manage three general classes of bias: those related to mapping the business intent into the AI implementation, those that arise due to the distribution of samples used for training, and those that are present in individual input samples. While there may be no simple or complete solution to this issue, best practices can be used to reduce the effects of bias on algorithmic outcomes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“While most research has focused on investigating and addressing bias in one-to-one interactions between humans and LLMs (18–20), less attention has been given to how these biases evolve through repeated communications in populations of LLM agents and, ultimately, in mixed human-LLM ecosystems (15), even though the safety of a single LLM does not necessarily imply the safety of a multi-agent system”
From Emergent social conventions and collective bias in LLM populations (2025) · cites this paper as [18]
In the citing paper
“Biases present in training data sets are known to lead to bias in subsequent AI models [12, 74].”
From “It’s Weird That it Knows What I Want”: Usability and Interactions with Copilot for Novice Programmers (2024) · cites this paper as [74]
Citations
Cites
Data Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Witten, I. H. (2008) · Cited by 9,143
(2018) · Cited by 3,510
Doyle, T. (2017) · The Information Society · doi:10.1080/01972243.2017.1354593 · Cited by 3,071
(2016) · Cited by 2,334
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1×
“bias [3] [4] and other categories of errors.”
Background(2011) · IEEE Software · doi:10.1109/ms.2011.67 · Cited by 1,989
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 3×
“Even seemingly innocuous input features can contain hidden biases [5, 9].”
BackgroundFriedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1×
“Other researchers have proposed taxonomies of bias that serve as an excellent starting point towards managing that bias [38, 39, 40] Cramer et.”
BackgroundAmazon Scraps Secret AI Recruiting Tool that Showed Bias against Women *
Dastin, J. (2022) · Ethics of Data and Analytics · doi:10.1201/9781003278290-44 · Cited by 1,165
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763 · cited 1×
“Other researchers have proposed taxonomies of bias that serve as an excellent starting point towards managing that bias [38, 39, 40] Cramer et.”
BackgroundSocial Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2016) · doi:10.2139/ssrn.2886526 · Cited by 763
Conference on Neural Information Processing Systems
Liu, L., Zhou, T., Long, G., Jiang, J., & Zhang, C. (2019) · UTS ePRESS (University of Technology Sydney) · Cited by 593
Artificial intelligence: the next digital frontier?
Bughin, J., Hazan, E., Ramaswamy, S., Chui, M., Allas, T., Dahlström, P., Henke, N., & Trench, M. (2017) · Cited by 569
Witten, I. H., Frank, E., & Hall, M. A. (2011) · Data Mining: Practical Machine Learning Tools and Techniques · doi:10.1016/b978-0-12-374856-0.00001-8 · Cited by 420
TFX: A TensorFlow-Based Production-Scale Machine Learning Platform
Baylor, D., Breck, E., Cheng, H.-T., Fiedel, N., Foo, C. Y., Haque, Z., Haykal, S., Ispir, M., Jain, V., Koc, L., et al. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098021 · Cited by 350
When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning AttacksPreprint
Suciu, O., Mărginean, R., Kaya, Y., Daumé, H., & Dumitraş, T. (2018) · arXiv (Cornell University) · Cited by 332
Kroll, J. A., Huey, J., Barocas, S., Felten, E. W., Reidenberg, J. R., Robinson, D. G., & Yu, H. (2017) · FLASH - Fordham Law Archive of Scholarship & History (Fordham University) · Cited by 299
Why is my classifier discriminatory?
Chen, I., Johansson, F. D., & Sontag, D. (2018) · arXiv · doi:10.48550/arxiv.1805.12002 · Cited by 185 · cited 1×
“have shown that supplementing training sets with additional data can be an effective strategy for reducing bias [24].”
BackgroundRunaway Feedback Loops in Predictive Policing
Ensign, D., Friedler, S. A., Neville, S., Scheidegger, C., & Venkatasubramanian, S. (2017) · arXiv · doi:10.48550/arxiv.1706.09847 · Cited by 161 · cited 2×
“For example, algorithms that predict crime based on police reports may cause more police to be deployed to the site of the predictions, which in turn may result in more police reports [7].”
BackgroundToward algorithmic transparency and accountability
Garfinkel, S., Matthews, J. N., Shapiro, S. S., & Smith, J. M. (2017) · Communications of the ACM · doi:10.1145/3125780 · Cited by 92
Self-citationAlgorithmic accountability: a primer
Donovan, J., Caplan, R., Matthews, J. N., & Hanson, L. (2018) · Cited by 91
Self-citationData and algorithmic bias in the web
Baeza-Yates, R. (2016) · Proceedings of the 8th ACM Conference on Web Science · doi:10.1145/2908131.2908135 · Cited by 84
Assessing and addressing algorithmic bias in practice
Cramer, H., Garcia-Gathright, J., Springer, A., & Reddy, S. (2018) · Interactions · doi:10.1145/3278156 · Cited by 52
Research works printed but not linked: 11
- Kahn , J. ( 2018 ). “ Artificial Intelligence Has Some Explaining to Do ”. Bloomberg Businessweek , 12 Dec 2018. Kahn, J. (2018). “Artificial Intelligence Has Some Explaining to Do”. Bloomberg Businessweek, 12 Dec 2018.
- 10.5555/3157382.3157584 · doi:10.5555/3157382.3157584
- Angwin , J. , Larson , J , Mattu , S. , and Kirchner , L . ( 2016 ). “ Machine Bias: There's software used across the country to predict future criminals. And it's biased against blacks ”. ProPublica , 23 May 2016. Angwin, J., Larson, J, Mattu, S., and Kirchner, L. (2016). “Machine Bias: There's software used across the country to predict future criminals. And it's biased against blacks”. ProPublica, 23 May 2016.
- Hao , K. ( 2019 ). “ This is How AI Bias Really Happens -— and Why It's so Hard to Fix ” . MIT Technology Review , 4 Feb 2019. Hao, K. (2019). “This is How AI Bias Really Happens -— and Why It's so Hard to Fix”. MIT Technology Review, 4 Feb 2019.
- Del Balso , M. and Hermann , J . ( 2017 ). “ Meet Michelangelo: Uber's Machine Learning Platform ”. Uber Engineering , 5 Sep 2017. Del Balso, M. and Hermann, J. (2017). “Meet Michelangelo: Uber's Machine Learning Platform”. Uber Engineering, 5 Sep 2017.
- Vincent , J. ( 2016 ). “ Twitter taught Microsoft's AI chatbot to be a racist asshole in less than a day ”. The Verge , 24 Mar 2016. Vincent, J. (2016). “Twitter taught Microsoft's AI chatbot to be a racist asshole in less than a day”. The Verge, 24 Mar 2016.
- Buolamwini , J. ( 2019 ). “ Aritificial Intelligence Has a Problem with Gender and Racial Bias ”. TIME , 7 Feb 2019. Buolamwini, J. (2019). “Aritificial Intelligence Has a Problem with Gender and Racial Bias”. TIME, 7 Feb 2019.
- Ghanta , S. and Talagala N . ( 2018 ). “ MLOps Health: Taking the Pulse of ML in Production ”. ITOpsTimes , 10 July 2018. Ghanta, S. and Talagala N. (2018). “MLOps Health: Taking the Pulse of ML in Production”. ITOpsTimes, 10 July 2018.
- ACM US Public Policy Council (USACM) ( 2017 ). Statement on Algorithmic Transparency and Accountability . January 12 , 2017 . https://www.acm.org/binaries/content/assets/public-policy/2017_usacm_statement_algorithms.pdf ACM US Public Policy Council (USACM) (2017). Statement on Algorithmic Transparency and Accountability. January 12, 2017. https://www.acm.org/binaries/content/assets/public-policy/2017_usacm_statement_algorithms.pdf
- Reisman , D. , Schultz , J. , Crawford , K. , and Whittaker , M . ( 2018 ). Algorithmic impact assessments: A practical framework for public agency accountability. AI Now institute . Apr. 2018. https://ainowinstitute.org/aiareport2018.pdf Reisman, D., Schultz, J., Crawford, K., and Whittaker, M. (2018). Algorithmic impact assessments: A practical framework for public agency accountability. AI Now institute. Apr. 2018. https://ainowinstitute.org/aiareport2018.pdf
- Caplan , R. , Donovan , J. , Hanson , L. , and Matthews , J . ( 2018 ). Algorithmic accountability primer . Data & Society . Apr. 2018. https://datasociety.net/wp-content/uploads/2018/04/Data_Society_Algorithmic_Accountability_Primer_FINAL-4.pdf Caplan, R., Donovan, J., Hanson, L., and Matthews, J. (2018). Algorithmic accountability primer. Data & Society. Apr. 2018. https://datasociety.net/wp-content/uploads/2018/04/Data_Society_Algorithmic_Accountability_Primer_FINAL-4.pdf
Cited by
Trustworthy Artificial Intelligence: A Review
Kaur, D., Uslu, S., Rittichier, K. J., & Durresi, A. (2023) · ACM Computing Surveys · doi:10.1145/3491209 · Cited by 709
“Other reasons for data bias can be mismatched datasets, unlearned data cases, and manipulated datasets [165].”
BackgroundBias in medical AI: Implications for clinical decision-making
Cross, J. L., Choma, M. A., & Onofrey, J. A. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000651 · Cited by 607
An Overview of Artificial Intelligence Ethics
Huang, C., Zhang, Z., Mao, B., & Yao, X. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3194503 · Cited by 482
Challenges and opportunities for artificial intelligence in auditing: Evidence from the field
Kokina, J., Blanchette, S., Davenport, T. H., & Pachamanova, D. (2025) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2025.100734 · Cited by 375
Generative AI for Customizable Learning Experiences
Pesovski, I., Santos, R., Henriques, R., & Trajkovik, V. (2024) · Sustainability · doi:10.3390/su16073034 · Cited by 274
“Mitigation strategies include the resampling and pre-processing of data used for training the models [68], using diverse and representative datasets [69], feedback loops [70], etc.”
Method
Metadata
- Authors
- Roselli, D., Matthews, J., & Talagala, N.
- Affiliations
- ParallelM
- Clarkson University (US)
- ORCID
- Matthews, Jeanna
- Venue
- Companion Proceedings of The 2019 World Wide Web Conference
- Publisher
- ACM
- Published
- 2019-05-13
- Pages
- 539-544
- OpenAlex
- W2946647999
- Cited by
- 237
- References
- 25
- Citations by year
- 2026: 34 · 2025: 61 · 2024: 53 · 2023: 49 · 2022: 24 · 2021: 12 · 2020: 3 · 2019: 1
- FWCI
- 10.43
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International World Wide Web Conference (WWW) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
- Keywords
- algorithmic bias, bias mitigation, AI governance, historical bias, training data distribution, explainability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.