The DOI matches its Crossref record; 264 of the 264 references it cites were found in a registry or an index; 2 of the 138 sources its claims cite were checked for support.
Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Open accessAbstract
Recently, large language models (LLMs) have attracted considerable attention due to their remarkable capabilities. However, LLMs’ generation of biased or hallucinatory content raised significant concerns, posing major challenges for their practical application. Many studies have dedicated efforts to address these critical issues, adopting various approaches to mitigate bias and hallucinations in LLM-generated content. Remarkably, no review papers have synthesized insights on these two primary problems. Addressing this gap, this paper aims to conduct a simultaneous and dual-focused review of the current landscape of research. The discussions encompass widely used and newly proposed benchmarks and evaluation methods on bias and hallucination in LLMs. This paper also investigates advanced mitigation methods and present a taxonomy based on different mitigation strategies. Moreover, a comparative analysis of the sources, mitigation methods, and evaluation methods for bias and hallucination is included. In the end, this paper provides a synthesis of current research trends and suggests potential directions for future research to address bias and hallucination in LLMs, considering the ongoing challenges in this field.
Overview
CoChat Summary
TL;DR
This review simultaneously surveys debiasing and dehallucinating in large language models, arguing that no prior review has synthesized these two trustworthiness problems together. It organizes evaluation benchmarks, mitigation methods, and a comparative analysis, then outlines challenges and future directions.
Key findings
- The paper claims no existing review comprehensively covers both bias and hallucination in LLM-generated content; it fills this gap with a dual-focused review of benchmarks, evaluation methods, and mitigation taxonomies.
- It proposes taxonomies: bias evaluation methods as embedding-based, MLM-based, and generation-based; hallucination evaluation as human, statistical-based, and model-based; mitigation methods by intervention stage (pre-processing, in-processing, intra-processing/post-processing).
- Comparative analysis: bias primarily stems from training data, while hallucination also arises from generation strategies, fine-tuning, model architecture, decoding algorithms, and exposure bias; bias evaluation is described as more mature and systematic than hallucination evaluation.
- Future directions include side effects of debiasing, more comprehensive bias measurement, multilingual/multicultural settings, using bias wisely, better hallucination evaluation, model editing, and RLHF limitations.
Methods
- Design: narrative literature review, not an empirical study; it states “No datasets were generated or analysed during the current study.”
- Data/sample: papers collected and filtered by titles and abstracts, with additional references drawn from collected papers; Figure 3 covers papers published between Mar 2022 and Oct 2023.
- Analysis: thematic synthesis and taxonomy construction for bias/hallucination definitions, causes, evaluation metrics/benchmarks, mitigation methods, plus comparative analysis and future directions.
Limitations
- Stated: the paper notes no datasets were generated or analysed; it also states that bias evaluation often targets specific categories and is insufficient for open-domain LLMs, and that automated hallucination evaluation still differs from human evaluation and varies across models/domains.
- Stated: current research mainly focuses on English-language models; multilingual and multicultural bias/hallucination remain challenging.
- Evident: as a review, it provides no primary experimental validation of the surveyed methods, and its literature coverage appears bounded by the search window ending Oct 2023.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
1 Valid · 10 Not valid · 93 Not checked · 68 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Some studies (Caliskan et al. 2017;Greenwald et al. 1998) have found that names associated with European Americans are more likely to be linked with pleasantness, while non-European names tend to be associated with unpleasantness.”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 3.1 Manifesting of bias
What the sources say
- Partly supportedcites Caliskan et al. 2017 sourceExperimental — accuracy not yet measured
In “Replicating Implicit Associations for Valence” (p. 4), the paper finds that European American names are more likely than African American names to be closer to pleasant than unpleasant words. The claim broadens that comparison to non-European names generally, which the reported test does not establish. (LLM, conf=0.87)
- Not checked yetcites Greenwald et al. 1998 source
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Most of the existing bias evaluation metrics have been for specific categories of bias, such as gender bias (Czarnowska et al. 2021).”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 6.1 Bias in LLMs
What the sources say
- Could not verifycites Czarnowska et al. 2021 sourceExperimental — accuracy not yet measured
The Introduction (p. 1) describes a survey of disparate fairness metrics, and Section 4 (p. 4) classifies 22 existing metrics. Neither passage says that most metrics were developed for specific bias categories such as gender. Searches for a gender-focused majority of existing metrics found examples of gender evaluation, but no statement establishing the claimed prevalence. (LLM, conf=0.70)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
None of the sources cited for this claim have been checked yet.
In the paper
“However, both papers similarly lack a systematic summary of debiasing methods and evaluation methods.”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > Table 1
What the sources say
- Not checked yetcites Ranaldi et al. 2023
- Not checked yetcites Meade et al. 2022
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“However, it has limitations as it cannot be applied to downstream tasks.”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 3.3.3 Post-processing for models
What the sources say
- Not checked yetcites Schick et al. (2021) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“However, Elazar and Goldberg (2018) point out that even after such training, traces of information can still linger in word embeddings.”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 3.3.2 In-processing during training
What the sources say
- Not checked yetcites Elazar and Goldberg (2018)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
60 more claims
Create a free account to see all 172 claims and to verify the 136 references still open.
- Probably not a substantial claim. 80
- It may state two separate claims as one. 54
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, LLMs are vulnerable to bias and hallucination, which depend predominantly on model training data and parameters.14”
From Evaluating the Quality and Safety of Ambient Digital Scribe Platforms Using Simulated Ambulatory Encounters (2025) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Consistency emerged as another problem, with participants reporting that the AI sometimes failed to maintain plot coherence”
From Game Plot Design With an LLM-Powered Assistant: An Empirical Study With Game Designers (2026) · cites this paper as Lin et al., 2024
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 4.4.3 Intra-processing without training
“The intra-processing methods without a training phase can be categorized into four key aspects: designing decode strategy, resorting to external knowledge, pre-detecting and preventing, and multi-agent interaction.”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > Resorting to external knowledge
“Ishibashi and Shimodaira (2023) fine-tunes the model so that it can generate harmless answers when answering questions that may involve sensitive information.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 5.3 Mitigation methods
“After the discussions in Sects.3.3 and 4.4, approaches to addressing bias and hallucination can be categorized into three main classes: pre-processing from data, in-processing during training, and post-processing during inference.”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 4.4 Dehallucinating methods
“We classify them into pre-processing for data, in-processing during training, intra-processing without training, and post-processing during inference, which are illustrated in Fig. 7.”
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Measuring individual differences in implicit cognition: the implicit association test
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.74.6.1464 · Cited by 9,210
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Research works printed but not linked: 26
- Banerjee S, Lavie A (2005) Meteor: an automatic metric for mt evaluation with improved correlation with human judgments. In: Proceedings of the ACL workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pp 65–72
- Bolukbasi T, Chang K-W, Zou JY, Saligrama V, Kalai AT (2016) Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Adv Neural Inf Process Syst 29
- Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877–1901
- Buolamwini J, Gebru T (2018) Gender shades: intersectional accuracy disparities in commercial gender classification. In: Conference on fairness, accountability and transparency. PMLR, pp 77–91
- Carlini N, Tramer F, Wallace E, Jagielski M, Herbert-Voss A, Lee K, Roberts A, Brown T, Song D, Erlingsson U et al. (2021) Extracting training data from large language models. In: 30th USENIX security symposium (USENIX Security 21), pp 2633–2650
- Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, Barham P, Chung HW, Sutton C, Gehrmann S et al (2023) Palm: scaling language modeling with pathways. J Mach Learn Res 24(240):1–113
- Dev S, Phillips J (2019) Attenuating bias in word vectors. In: The 22nd international conference on artificial intelligence and statistics. PMLR, pp 879–887
- Dinan E, Roller S, Shuster K, Fan A, Auli M, Weston J (2018) Wizard of Wikipedia: knowledge-powered conversational agents. In: International conference on learning representations
- Gao L, Schulman J, Hilton J (2023) Scaling laws for reward model overoptimization. In: Krause A, Brunskill E, Cho K, Engelhardt B, Sabato S, Scarlett J (eds.), Proceedings of the 40th international conference on machine learning research, vol. 202. PMLR, pp 10835–10866. https://proceedings.mlr.press/v202/gao23h.html
- Honovich O, Choshen L, Aharoni R, Neeman E, Szpektor I, Abend O (2021) Q2: evaluating factual consistency in knowledge-grounded dialogues via question generation and question answering. In: Proceedings of the 2021 conference on empirical methods in natural language processing, pp 7856–7870 · DOI · Paper page
- Le Bras R, Swayamdipta S, Bhagavatula C, Zellers R, Peters M, Sabharwal A, Choi Y (2020) Adversarial filters of dataset biases. International conference on machine learning. PMLR, pp 1078–1088
- Lee N, Ping W, Xu P, Patwary M, Fung PN, Shoeybi M, Catanzaro B (2022) Factuality enhanced language models for open-ended text generation. Adv Neural Inf Process Syst 35:34586–34599
- Lin S, Hilton J, Evans O (2022) Truthfulqa: measuring how models mimic human falsehoods. In: Proceedings of the 60th annual meeting of the association for computational linguistics, vol. 1 (long papers), pp 3214–3252 · DOI · Paper page
- McFadden AC, Marsh GE, Price BJ, Hwang Y (1992) A study of race and gender bias in the punishment of school children. Educ Treat Child 15(2):140–146
- Meng K, Bau D, Andonian A, Belinkov Y (2022) Locating and editing factual associations in GPT. Adv Neural Inf Process Syst 35:17359–17372
- Ntoutsi E, Fafalios P, Gadiraju U, Iosifidis V, Nejdl W, Vidal M-E, Ruggieri S, Turini F, Papadopoulos S, Krasanakis E et al (2020) Bias in data-driven artificial intelligence systems—an introductory survey. Wiley Interdisc Rev: Data Min Knowl Discov 10(3):1356
- Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A et al (2022) Training language models to follow instructions with human feedback. Adv Neural Inf Process Syst 35:27730–27744
- Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I et al (2019) Language models are unsupervised multitask learners. OpenAI blog 1(8):9
- Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(1):5485–5551
- Sahlgren M, Olsson F (2019) Gender bias in pretrained Swedish embeddings. In: Proceedings of the 22nd Nordic conference on computational linguistics, pp 35–43
- Santhanam S, Hedayatnia B, Gella S, Padmakumar A, Kim S, Liu Y, Hakkani-Tür D (2021) Rome was built in 1776: a case study on factual correctness in knowledge-grounded response generation. In: EMNLP 2021 workshop on NLP for conversational AI
- Schmidt B (2015) Rejecting the gender binary: a vector-space operation. Ben’s Bookworm Blog
- Tan YC, Celis LE (2019) Assessing social and intersectional biases in contextualized word representations. Adv Neural Inf Process Syst 32
- Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30
- Zaheri S, Leath J, Stroud D (2020) Toxic comment classification. SMU Data Sci Rev 3(1):13
- Zhao T, Wei M, Preston JS, Poon H (2023) Automatic calibration and error correction for large language models via pareto optimal self-supervision. arXiv preprint arXiv:2306.16564 · arXiv
Cited by
Bias in Large Language Models: Origin, Evaluation, and Mitigation
Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2026) · Electronics · doi:10.3390/electronics15091824 · Cited by 247
Bias in Large Language Models: Origin, Evaluation, and Mitigation
Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2024) · arXiv · doi:10.48550/arxiv.2411.10915 · Cited by 208
A survey on multilingual large language models: corpora, alignment, and bias
Xu, Y., Hu, L., Zhao, J., Qiu, Z., Xu, K., Ye, Y., & Gu, H. (2025) · Frontiers of Computer Science · doi:10.1007/s11704-024-40579-4 · Cited by 223
Evaluating Trustworthiness in AI: Risks, Metrics, and Applications Across Industries
Nastoska, A., Jancheska, B., Rizinski, M., & Trajanov, D. (2025) · Electronics · doi:10.3390/electronics14132717 · Cited by 167
Abdel‐Rehim, A., Zenil, H., Orhobor, O. I., Fisher, M., Collins, R. J., Bourne, E., Fearnley, G. W., Tate, E., Smith, H. X., Soldatova, L., et al. (2025) · Journal of The Royal Society Interface · doi:10.1098/rsif.2024.0674 · Cited by 55
Metadata
- Authors
- Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H.
- Affiliations
- Beijing Institute of Technology (CN)
- Corresponding author
- Zhang, Huiyan
- Venue
- Artificial Intelligence Review
- Published
- 2024-08-10
- Volume
- 57
- Issue
- 9
- ISSN
- 0269-2821 (ISSN-L)
- 1573-7462
- OpenAlex
- W4401484139
- Cited by
- 141
- References
- 216
- Citations by year
- 2026: 70 · 2025: 69 · 2024: 2
- FWCI
- 58.44
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- Artificial Intelligence Review (publishedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, bias mitigation, hallucination mitigation, trustworthy AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models. https://app.cochat.ai/papers/10.1007/s10462-024-10896-y
- The paper
- Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024). Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models. Artificial Intelligence Review. https://doi.org/10.1007/s10462-024-10896-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.