CoChat

Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models

Open access
Lin, Zichao, Guan, Shuyan, Zhang, Wending, Zhang, Huiyan, Li, Yugang, Zhang, Huaping · 2024 · Artificial Intelligence Review

The DOI matches its Crossref record; 205 of the 269 references it cites were found in a registry or an index; 2 of the 138 sources its claims cite were checked for support.

Abstract

Recently, large language models (LLMs) have attracted considerable attention due to their remarkable capabilities. However, LLMs’ generation of biased or hallucinatory content raised significant concerns, posing major challenges for their practical application. Many studies have dedicated efforts to address these critical issues, adopting various approaches to mitigate bias and hallucinations in LLM-generated content. Remarkably, no review papers have synthesized insights on these two primary problems. Addressing this gap, this paper aims to conduct a simultaneous and dual-focused review of the current landscape of research. The discussions encompass widely used and newly proposed benchmarks and evaluation methods on bias and hallucination in LLMs. This paper also investigates advanced mitigation methods and present a taxonomy based on different mitigation strategies. Moreover, a comparative analysis of the sources, mitigation methods, and evaluation methods for bias and hallucination is included. In the end, this paper provides a synthesis of current research trends and suggests potential directions for future research to address bias and hallucination in LLMs, considering the ongoing challenges in this field.

Overview

CoChat Summary

TL;DR

This review simultaneously surveys debiasing and dehallucinating in large language models, arguing that no prior review has synthesized these two trustworthiness problems together. It organizes evaluation benchmarks, mitigation methods, and a comparative analysis, then outlines challenges and future directions.

Key findings

  • The paper claims no existing review comprehensively covers both bias and hallucination in LLM-generated content; it fills this gap with a dual-focused review of benchmarks, evaluation methods, and mitigation taxonomies.
  • It proposes taxonomies: bias evaluation methods as embedding-based, MLM-based, and generation-based; hallucination evaluation as human, statistical-based, and model-based; mitigation methods by intervention stage (pre-processing, in-processing, intra-processing/post-processing).
  • Comparative analysis: bias primarily stems from training data, while hallucination also arises from generation strategies, fine-tuning, model architecture, decoding algorithms, and exposure bias; bias evaluation is described as more mature and systematic than hallucination evaluation.
  • Future directions include side effects of debiasing, more comprehensive bias measurement, multilingual/multicultural settings, using bias wisely, better hallucination evaluation, model editing, and RLHF limitations.

Methods

  • Design: narrative literature review, not an empirical study; it states “No datasets were generated or analysed during the current study.”
  • Data/sample: papers collected and filtered by titles and abstracts, with additional references drawn from collected papers; Figure 3 covers papers published between Mar 2022 and Oct 2023.
  • Analysis: thematic synthesis and taxonomy construction for bias/hallucination definitions, causes, evaluation metrics/benchmarks, mitigation methods, plus comparative analysis and future directions.

Limitations

  • Stated: the paper notes no datasets were generated or analysed; it also states that bias evaluation often targets specific categories and is insufficient for open-domain LLMs, and that automated hallucination evaluation still differs from human evaluation and varies across models/domains.
  • Stated: current research mainly focuses on English-language models; multilingual and multicultural bias/hallucination remain challenging.
  • Evident: as a review, it provides no primary experimental validation of the surveyed methods, and its literature coverage appears bounded by the search window ending Oct 2023.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s10462-024-10896-y

Cites

205 of 269 references verified0 citation errors · 64 not checkable

Statistics

Nothing to recompute.

Figures

Image 1
CC-BY-NC-ND · From the PDF
Fig. 2 A guide that lays out the sections of the review paper
Fig. 2 A guide that lays out the sections of the review paperCC-BY-NC-ND · From the PDF
Fig. 3 The number of papers on bias and hallucination between Mar 2022 and Oct 2023
Fig. 3 The number of papers on bias and hallucination between Mar 2022 and Oct 2023CC-BY-NC-ND · From the PDF
Fig. 4 Examples of three classes of bias evaluation methods
Fig. 4 Examples of three classes of bias evaluation methodsCC-BY-NC-ND · From the PDF
Fig. 5 Illustration of our taxonomy of mitigation states for debiasing
Fig. 5 Illustration of our taxonomy of mitigation states for debiasingCC-BY-NC-ND · From the PDF
Fig. 6 Examples of three classes of hallucination evaluation methods
Fig. 6 Examples of three classes of hallucination evaluation methodsCC-BY-NC-ND · From the PDF
Fig. 7 Illustration of our taxonomy of mitigation states for dehallucinating
Fig. 7 Illustration of our taxonomy of mitigation states for dehallucinatingCC-BY-NC-ND · From the PDF

Verification

Claims the paper makes

1 Valid · 10 Not valid · 93 Not checked · 68 Own results and interpretations

60 more claims

Create a free account to see all 172 claims and to verify the 136 references still open.

See all claims — free account
  • Probably not a substantial claim. 80
  • It may state two separate claims as one. 54

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Items 201–269 of 269

Research works printed but not linked: 30

  1. Banerjee S, Lavie A (2005) Meteor: an automatic metric for mt evaluation with improved correlation with human judgments. In: Proceedings of the ACL workshop on intrinsic and extrinsic evaluation meas- ures for machine translation and/or summarization, pp 65-72
  2. Bolukbasi T, Chang K-W, Zou JY, Saligrama V, Kalai AT (2016) Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Adv Neural Inf Process Syst 29
  3. Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877-1901
  4. Buolamwini J, Gebru T (2018) Gender shades: intersectional accuracy disparities in commercial gender classification. In: Conference on fairness, accountability and transparency. PMLR, pp 77-91
  5. Cao Y, Kang Y, Sun L (2023) Instruction mining: High-quality instruction data selection for large lan- guage models. arXiv preprint arXiv: 2307. 06290
  6. Carlini N, Tramer F, Wallace E, Jagielski M, Herbert-Voss A, Lee K, Roberts A, Brown T, Song D, Erlingsson U et al. (2021) Extracting training data from large language models. In: 30th USENIX security symposium (USENIX Security 21), pp 2633-2650
  7. Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, Barham P, Chung HW, Sutton C, Gehrmann S et al (2023) Palm: scaling language modeling with pathways. J Mach Learn Res 24(240):1-113
  8. Dev S, Phillips J (2019) Attenuating bias in word vectors. In: The 22nd international conference on arti- ficial intelligence and statistics. PMLR, pp 879-887
  9. Dinan E, Roller S, Shuster K, Fan A, Auli M, Weston J (2018) Wizard of Wikipedia: knowledge-pow- ered conversational agents. In: International conference on learning representations
  10. Gao L, Schulman J, Hilton J (2023) Scaling laws for reward model overoptimization. In: Krause A, Brunskill E, Cho K, Engelhardt B, Sabato S, Scarlett J (eds.), Proceedings of the 40th interna- tional conference on machine learning research, vol. 202. PMLR, pp 10835-10866. https:// proce edings. mlr. press/ v202/ gao23h. html
  11. Honovich O, Choshen L, Aharoni R, Neeman E, Szpektor I, Abend O (2021) Q2: evaluating factual consist- ency in knowledge-grounded dialogues via question generation and question answering. In: Proceed- ings of the 2021 conference on empirical methods in natural language processing, pp 7856-7870 · DOI · Paper page
  12. Kabir S, Udo-Imeh DN, Kou B, Zhang T (2023) Who answers it better? An in-depth analysis of chatgpt and stack overflow answers to software engineering questions. arXiv preprint arXiv: 2308. 02312
  13. Le Bras R, Swayamdipta S, Bhagavatula C, Zellers R, Peters M, Sabharwal A, Choi Y (2020) Adversarial filters of dataset biases. International conference on machine learning. PMLR, pp 1078-1088
  14. Lee N, Ping W, Xu P, Patwary M, Fung PN, Shoeybi M, Catanzaro B (2022) Factuality enhanced language models for open-ended text generation. Adv Neural Inf Process Syst 35:34586-34599
  15. Liu F, Lin K, Li L, Wang J, Yacoob Y, Wang L (2023) Aligning large multi-modal model with robust instruction tuning. arXiv preprint arXiv: 2306. 14565
  16. McFadden AC, Marsh GE, Price BJ, Hwang Y (1992) A study of race and gender bias in the punishment of school children. Educ Treat Child 15(2):140-146
  17. Meng K, Bau D, Andonian A, Belinkov Y (2022) Locating and editing factual associations in GPT. Adv Neural Inf Process Syst 35:17359-17372
  18. Ntoutsi E, Fafalios P, Gadiraju U, Iosifidis V, Nejdl W, Vidal M-E, Ruggieri S, Turini F, Papadopoulos S, Krasanakis E et al (2020) Bias in data-driven artificial intelligence systems-an introductory survey. Wiley Interdisc Rev: Data Min Knowl Discov 10(3):1356
  19. Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A et al (2022) Training language models to follow instructions with human feedback. Adv Neural Inf Process Syst 35:27730-27744
  20. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I et al (2019) Language models are unsuper- vised multitask learners. OpenAI blog 1(8):9
  21. Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Explor- ing the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(1):5485-5551
  22. Sahlgren M, Olsson F (2019) Gender bias in pretrained Swedish embeddings. In: Proceedings of the 22nd Nordic conference on computational linguistics, pp 35-43
  23. Santhanam S, Hedayatnia B, Gella S, Padmakumar A, Kim S, Liu Y, Hakkani-Tür D (2021) Rome was built in 1776: a case study on factual correctness in knowledge-grounded response generation. In: EMNLP 2021 workshop on NLP for conversational AI
  24. Schmidt B (2015) Rejecting the gender binary: a vector-space operation. Ben’s Bookworm Blog
  25. Sun T, Zhang X, He Z, Li P, Cheng Q, Yan H, Liu X, Shao Y, Tang Q, Zhao X, et al. (2023) Moss: training conversational language models from synthetic data 7. arXiv preprint arXiv: 2307. 15020
  26. Tan YC, Celis LE (2019) Assessing social and intersectional biases in contextualized word representations. Adv Neural Inf Process Syst 32
  27. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Atten- tion is all you need. Adv Neural Inf Process Syst 30
  28. Wiener N (1950) The human use of human beings: Cybernetics and society Workshop B, Scao TL, Fan A, Akiki C, Pavlick E, Ilić S, Hesslow D, Castagné R, Luccioni AS, Yvon F et al. (2022) Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv: 2211. 05100
  29. Zaheri S, Leath J, Stroud D (2020) Toxic comment classification. SMU Data Sci Rev 3(1):13
  30. Zhao T, Wei M, Preston JS, Poon H (2023) Automatic calibration and error correction for large language models via pareto optimal self-supervision. arXiv preprint arXiv: 2306. 16564

Cited by

See all 188

Metadata

Authors
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H.
Affiliations
Beijing Institute of Technology (CN)
Corresponding author
Zhang, Huiyan
Venue
Artificial Intelligence Review
Published
2024-08-10
Volume
57
Issue
9
ISSN
  • 0269-2821 (ISSN-L)
  • 1573-7462
OpenAlex
W4401484139
Cited by
141
References
216
Citations by year
2026: 70 · 2025: 69 · 2024: 2
FWCI
58.44
Type
journal article
Language
en
Open access
Licence
Locations
Artificial Intelligence Review (publishedVersion, cc-by-nc-nd)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
large language models, bias mitigation, hallucination mitigation, trustworthy AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.