Knowledge-Enhanced Language Models Are Not Bias-Proof: Situated Knowledge and Epistemic Injustice in AI
Open accessAbstract
The factual inaccuracies ("hallucinations") of large language models have recently inspired more research on knowledge-enhanced language modeling approaches. These are often assumed to enhance the overall trustworthiness and objectivity of language models. Meanwhile, the issue of bias is usually only mentioned as a limitation of statistical representations. This dissociation of knowledge-enhancement and bias is in line with previous research on AI engineers’ assumptions about knowledge, which indicate that knowledge is commonly understood as objective and value-neutral by this community. We argue that claims and practices by actors of the field still reflect this underlying conception of knowledge. We contrast this assumption with literature from social and, in particular, feminist epistemology, which argues that the idea of a universal disembodied knower is blind to the reality of knowledge practices and seriously challenges claims of "objective" or "neutral" knowledge.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The discipline of machine learning has construed knowledge as universal and readily gleanable from observable patterns in data, thereby enforcing the view from nowhere via the attribution of objectivity to the systems' outputs.
“As Kraft & Soulier [23] argue, the discipline of machine learning has construed knowledge as universal and readily gleanable from observable patterns in data, thereby enforcing the view from nowhere via the attribution of objectivity to the systems' outputs.”
From A taxonomy of epistemic injustice in the context of AI and the case for generative hermeneutical erasure (2025) · cites this paper as Kraft & Soulier [23]
Cites
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Epistemic Injustice: Power and the Ethics of Knowing
Fricker, M. (2007) · doi:10.1093/acprof:oso/9780198237907.001.0001 · Cited by 9,198 · see citing works
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · see citing works
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · cited 1× · see citing works
“Besides aforementioned efforts to facilitate more diverse contribution, we also need transparent documentation practices that allow scrutiny of knowledge bases and their original knowers [7, 27].”
BackgroundConceptNet 5.5: An Open Multilingual Graph of General Knowledge
Speer, R., Chin, J., & Havasi, C. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11164 · Cited by 2,312 · see citing works
Longino, H. E. (2002) · doi:10.1515/9780691187013 · Cited by 1,622 · see citing works
Unifying Large Language Models and Knowledge Graphs: A Roadmap
Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., & Wu, X. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3352100 · Cited by 1,376 · see citing works
Friedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1× · see citing works
“Social bias is observed when language models " systematically and unfairly discriminate against certain individuals or groups of individuals in favor of others" [23, p. 332].”
BackgroundRetrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876 · see citing works
Bender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854 · cited 1× · see citing works
“Besides aforementioned efforts to facilitate more diverse contribution, we also need transparent documentation practices that allow scrutiny of knowledge bases and their original knowers [7, 27].”
BackgroundMason, R. (2011) · Hypatia · doi:10.1111/j.1527-2001.2011.01175.x · Cited by 265 · see citing works
Language Models (Mostly) Know What They Know
Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Hatfield-Dodds, Z., DasSarma, N., Tran-Johnson, E., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05221 · Cited by 187 · see citing works
To protect science, we must use LLMs as zero-shot translators
Mittelstadt, B., Wachter, S., & Russell, C. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01744-0 · Cited by 58 · see citing works
Discovering Latent Knowledge in Language Models Without Supervision
Burns, C., Ye, H., Klein, D., & Steinhardt, J. (2022) · arXiv · doi:10.48550/arxiv.2212.03827 · Cited by 55 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Mollema, W. J. T. (2025) · AI and Ethics · doi:10.1007/s43681-025-00801-w · Cited by 24 · see citing works
“As Kraft & Soulier [23] argue, the discipline of machine learning has construed knowledge as universal and readily gleanable from observable patterns in data, thereby enforcing the view from nowhere via the attribution of objectivity to the syst ems’ outputs.”
Olaniyan, Y. D., Martins, M. O., & Al Maqrashi, R. H. (2026) · Frontiers in Human Dynamics · doi:10.3389/fhumd.2026.1790324 · Cited by 8 · see citing works
- Authors
- Kraft, A., & Soulier, E.
- Affiliations
- Leuphana University of Lüneburg (DE)
- Universität Hamburg (DE)
- Venue
- The 2024 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2024-06-03
- Pages
- 1433-1445
- OpenAlex
- W4399353366
- Cited by
- 25
- References
- 17
- Citations by year
- 2026: 10 · 2025: 15
- FWCI
- 11.41
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- Multilingual Matters (Channel View Publications) (submittedVersion, cc-by)
- Funders
- German Research Foundation (DFG) (460234259)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Feminist Epistemology and Gender Studies
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Keywords
- knowledge-enhanced language models, large language models, factual hallucination, algorithmic bias, situated knowledge, epistemic injustice, feminist epistemology, objectivity, knowledge practices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
