Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 89 of the 89 references it cites were found in a registry or an index; 7 of the 90 sources its claims cite were checked for support.

AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War

Open access
Humphreys, Declan · 2025 · Philosophy & Technology

Abstract

Generative AI is increasingly used as an epistemic tool to aid inquiry, broaden our knowledge and generally help us find things out. It is now embedded in internet search functions to condense information and answer user questions without further need to access websites or news articles. However, models based on generative AI come with inherent flaws which may inhibit their effectiveness as tools to aid responsible inquiry. While the risk of misinformation due to ‘hallucinations’ has been documented, this paper instead focuses on the capacity for these models to present moral or political bias reflective of the preferences of the model designers and crowd worker labellers. This paper takes the novel approach of tracing the ethical issue of bias back to fundamental alignment problem within the process of Reinforced Learning from Human Feedback (RLHF), a key step in training generative AI models. The potential for language models to be influenced by their designers, censored to certain topics, or trained to be more politically leaning one way or another has the capability of insidiously causing epistemic harm. By applying the concepts of bias of crowds to the training process of language models I show how they can impede effective epistemic inquiry and cause epistemic and moral harm. In doing so I present an argument showing that the ethics of AI language models cannot be removed from the socio-technical systems from which they are created.

Overview

CoChat Summary

## TL;DR

This paper argues that large language models are epistemic tools with important limitations, and that RLHF can import collective, institutional, and implicit biases into their outputs. It claims these biases can produce epistemic harm and epistemic injustice, especially when models are used for information, judgment, or political guidance.

## Key findings

  • RLHF reduces toxicity and improves helpfulness, but it does not reliably eliminate bias; the paper notes that Ouyang et al. (2022) found RLHF did not significantly reduce bias, and that the model could still make simple mistakes.
  • In one comparison discussed from Santurkar et al. (2023), OpenAI’s RLHF-tuned “DaVinci” models were less aligned with the general populace than the base model; the tuned model’s views aligned more with people who were “liberal, high income, well-educated, and not religious,” matching the annotator pool described as predominantly young Southeast Asian and White with a college degree.
  • The paper reports inter-labeller agreement of about 73% in Ouyang et al. (2022), and elsewhere says agreement in RLHF has been reported at 63–77%, highlighting substantial disagreement in human preference data.
  • Hofmann et al. (2024) is cited as showing covert racial bias: models associated African Americans with negative attributes such as “lazy” and “dirty,” and when asked to judge job fit or murder cases, LLMs gave less-prestigious jobs and more death-penalty judgments for AAE than SAE inputs.

## Methods

  • Conceptual/philosophical argument rather than an empirical study; the paper synthesizes prior technical and social-scientific literature on RLHF, bias, and epistemic harm.
  • Uses examples and findings from prior studies on hallucinations, political bias, sycophancy, labeller disagreement, and implicit bias in LLMs to build its argument.
  • The main analysis traces bias in LLMs back to RLHF and the socio-technical conditions of labelling, organizational control, and crowd-based aggregation.

## Limitations

  • The paper states that it does not argue RLHF is the only cause of bias in LLMs, only that it is underexplored as a source of ethical and philosophical concern.
  • It notes that some proposed remedies, such as measuring implicit bias or using psychological tests adapted from humans, face difficulties when applied directly to LLMs.
  • Evident from the text: this is a review/argument paper and does not present new empirical data of its own.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s13347-025-00928-y

Cites

89 of 89 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

7 Valid · 1 Not valid · 67 Not checked · 74 Own results and interpretations

67 more claims

Create a free account to see all 149 claims and to verify the 83 references still open.

See all claims — free account
  • It may state two separate claims as one. 33
  • Probably not a substantial claim. 60

Create a free account to see these claims.

Citations

Cites

Show all 72 references

Research works printed but not linked: 3

  1. Halavais, A. (2017). Search engine society. Wiley.
  2. Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., & Hashimoto, T. (2023). Whose opinions do language models reflect? International Conference on Machine Learning.
  3. Wei, A., Haghtalab, N., & Steinhardt, J. (2024). Jailbroken: How does llm safety training fail? Advances in neural information processing systems, 36.

Cited by

See all 26

Metadata

Authors
Humphreys, D.
Affiliations
University of the Sunshine Coast (AU)
Corresponding author
Humphreys, Declan
Venue
Philosophy & Technology
Published
2025-07-11
Volume
38
Issue
3
ISSN
  • 2210-5433 (ISSN-L)
  • 2210-5441
OpenAlex
W4412188302
Cited by
26
References
72
Citations by year
2026: 24 · 2025: 2
FWCI
27.18
Type
journal article
Language
en
Open access
Licence
Locations
Philosophy & Technology (publishedVersion, cc-by)
Funders
University of the Sunshine Coast
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Sociology and Political Science › Feminist Epistemology and Gender Studies
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
Keywords
epistemic harm, generative AI, reinforcement learning from human feedback, AI alignment, crowd workers, AI ethics, hallucinations, socio-technical systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War. https://app.cochat.ai/papers/10.1007/s13347-025-00928-y
The paper
Humphreys, D. (2025). AI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War. Philosophy & Technology. https://doi.org/10.1007/s13347-025-00928-y

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.