Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Open accessThe DOI matches its Crossref record; 205 of the 269 references it cites were found in a registry or an index; 2 of the 138 sources its claims cite were checked for support.
Abstract
Recently, large language models (LLMs) have attracted considerable attention due to their remarkable capabilities. However, LLMs’ generation of biased or hallucinatory content raised significant concerns, posing major challenges for their practical application. Many studies have dedicated efforts to address these critical issues, adopting various approaches to mitigate bias and hallucinations in LLM-generated content. Remarkably, no review papers have synthesized insights on these two primary problems. Addressing this gap, this paper aims to conduct a simultaneous and dual-focused review of the current landscape of research. The discussions encompass widely used and newly proposed benchmarks and evaluation methods on bias and hallucination in LLMs. This paper also investigates advanced mitigation methods and present a taxonomy based on different mitigation strategies. Moreover, a comparative analysis of the sources, mitigation methods, and evaluation methods for bias and hallucination is included. In the end, this paper provides a synthesis of current research trends and suggests potential directions for future research to address bias and hallucination in LLMs, considering the ongoing challenges in this field.
Overview
CoChat Summary
TL;DR
This review simultaneously surveys debiasing and dehallucinating in large language models, arguing that no prior review has synthesized these two trustworthiness problems together. It organizes evaluation benchmarks, mitigation methods, and a comparative analysis, then outlines challenges and future directions.
Key findings
- The paper claims no existing review comprehensively covers both bias and hallucination in LLM-generated content; it fills this gap with a dual-focused review of benchmarks, evaluation methods, and mitigation taxonomies.
- It proposes taxonomies: bias evaluation methods as embedding-based, MLM-based, and generation-based; hallucination evaluation as human, statistical-based, and model-based; mitigation methods by intervention stage (pre-processing, in-processing, intra-processing/post-processing).
- Comparative analysis: bias primarily stems from training data, while hallucination also arises from generation strategies, fine-tuning, model architecture, decoding algorithms, and exposure bias; bias evaluation is described as more mature and systematic than hallucination evaluation.
- Future directions include side effects of debiasing, more comprehensive bias measurement, multilingual/multicultural settings, using bias wisely, better hallucination evaluation, model editing, and RLHF limitations.
Methods
- Design: narrative literature review, not an empirical study; it states “No datasets were generated or analysed during the current study.”
- Data/sample: papers collected and filtered by titles and abstracts, with additional references drawn from collected papers; Figure 3 covers papers published between Mar 2022 and Oct 2023.
- Analysis: thematic synthesis and taxonomy construction for bias/hallucination definitions, causes, evaluation metrics/benchmarks, mitigation methods, plus comparative analysis and future directions.
Limitations
- Stated: the paper notes no datasets were generated or analysed; it also states that bias evaluation often targets specific categories and is insufficient for open-domain LLMs, and that automated hallucination evaluation still differs from human evaluation and varies across models/domains.
- Stated: current research mainly focuses on English-language models; multilingual and multicultural bias/hallucination remain challenging.
- Evident: as a review, it provides no primary experimental validation of the surveyed methods, and its literature coverage appears bounded by the search window ending Oct 2023.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
1 Valid · 10 Not valid · 93 Not checked · 68 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Some studies (Caliskan et al. 2017;Greenwald et al. 1998) have found that names associated with European Americans are more likely to be linked with pleasantness, while non-European names tend to be associated with unpleasantness.”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 3.1 Manifesting of bias
What the sources say
- Partly supportedcites Caliskan et al. 2017 matched to “Semantics derived automatically from language corpora contain human-like biases” (2017) sourceExperimental — accuracy not yet measured
In “Replicating Implicit Associations for Valence” (p. 4), the paper finds that European American names are more likely than African American names to be closer to pleasant than unpleasant words. The claim broadens that comparison to non-European names generally, which the reported test does not establish. (LLM, conf=0.87)
- Not checked yetcites Greenwald et al. 1998 matched to “Measuring individual differences in implicit cognition: the implicit association test” (1998) source
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Most of the existing bias evaluation metrics have been for specific categories of bias, such as gender bias (Czarnowska et al. 2021).”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 6.1 Bias in LLMs
What the sources say
- Could not verifycites Czarnowska et al. 2021 matched to “Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics” (2021) sourceExperimental — accuracy not yet measured
The Introduction (p. 1) describes a survey of disparate fairness metrics, and Section 4 (p. 4) classifies 22 existing metrics. Neither passage says that most metrics were developed for specific bias categories such as gender. Searches for a gender-focused majority of existing metrics found examples of gender evaluation, but no statement establishing the claimed prevalence. (LLM, conf=0.70)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
None of the sources cited for this claim have been checked yet.
In the paper
“However, both papers similarly lack a systematic summary of debiasing methods and evaluation methods.”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > Table 1
What the sources say
- Not checked yetcites Ranaldi et al. 2023
- Not checked yetcites Meade et al. 2022
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“However, it has limitations as it cannot be applied to downstream tasks.”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 3.3.3 Post-processing for models
What the sources say
- Not checked yetcites Schick et al. (2021) matched to “Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP” (2021) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“However, Elazar and Goldberg (2018) point out that even after such training, traces of information can still linger in word embeddings.”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 3.3.2 In-processing during training
What the sources say
- Not checked yetcites Elazar and Goldberg (2018)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
60 more claims
Create a free account to see all 172 claims and to verify the 136 references still open.
- Probably not a substantial claim. 80
- It may state two separate claims as one. 54
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, LLMs are vulnerable to bias and hallucination, which depend predominantly on model training data and parameters.14”
From Evaluating the Quality and Safety of Ambient Digital Scribe Platforms Using Simulated Ambulatory Encounters (2025) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Consistency emerged as another problem, with participants reporting that the AI sometimes failed to maintain plot coherence”
From Game Plot Design With an LLM-Powered Assistant: An Empirical Study With Game Designers (2026) · cites this paper as Lin et al., 2024
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 4.4.3 Intra-processing without training
“The intra-processing methods without a training phase can be categorized into four key aspects: designing decode strategy, resorting to external knowledge, pre-detecting and preventing, and multi-agent interaction.”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > Resorting to external knowledge
“Ishibashi and Shimodaira (2023) fine-tunes the model so that it can generate harmless answers when answering questions that may involve sensitive information.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 5.3 Mitigation methods
“After the discussions in Sects.3.3 and 4.4, approaches to addressing bias and hallucination can be categorized into three main classes: pre-processing from data, in-processing during training, and post-processing during inference.”
§ Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models > 4.4 Dehallucinating methods
“We classify them into pre-processing for data, in-processing during training, intra-processing without training, and post-processing during inference, which are illustrated in Fig. 7.”
Citations
Cites
Items 201–269 of 269
Linguistically-Informed Transformations (LIT): A Method for Automatically Generating Contrast Sets
Li, C., Shengshuo, L., Liu, Z., Wu, X., Zhou, X., & Steinert‐Threlkeld, S. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.12 · Cited by 47
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark
Dziri, N., Rashkin, H., Linzen, T., & Reitter, D. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00506 · Cited by 47
Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation
Mündler, N., He, J., Jenko, S., & Vechev, M. (2023) · arXiv · doi:10.48550/arxiv.2305.15852 · Cited by 47
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark
Dziri, N., Rashkin, H., Linzen, T., & Reitter, D. (2021) · arXiv · doi:10.48550/arxiv.2105.00071 · Cited by 47
Unmasking the Mask – Evaluating Social Biases in Masked Language Models
Kaneko, M., & Bollegala, D. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21453 · Cited by 46
Retrieval-guided Counterfactual Generation for QA
Paranjape, B., Lamm, M., & Tenney, I. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.117 · Cited by 46
Zero-Resource Hallucination Prevention for Large Language ModelsPreprint
Luo, J., Xiao, C., & Ma, F. (2023) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2309.02654 · Cited by 46
Specialist or Generalist? Instruction Tuning for Specific NLP Tasks
Shi, C., Su, Y., Yang, C. H., Yang, Y., & Cai, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.947 · Cited by 45
PandaGPT: One Model To Instruction-Follow Them All
Su, Y., Lan, T., Li, H., Xu, J., Wang, Y., & Cai, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.16355 · Cited by 45
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
Yüksekgönül, M., Chandrasekaran, V., Jones, E., Gunasekar, S., Naik, R., Palangi, H., Kamar, E. S., & Nushi, B. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2309.15098 · Cited by 45
Debiasing Embeddings for Reduced Gender Bias in Text Classification
Prost, F., Thain, N., & Bolukbasi, T. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3810 · Cited by 44
Yang, Y., Li, H., Wang, Y., & Wang, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.04782 · Cited by 44
Chain-of-Verification Reduces Hallucination in Large Language Models
Dhuliawala, S., Komeili, M., Xu, J., Raileanu, R., Li, X., Celikyilmaz, A., & Weston, J. (2023) · arXiv · doi:10.48550/arxiv.2309.11495 · Cited by 42
Tailor: Generating and Perturbing Text with Semantic Controls
Ross, A., Wu, T., Peng, H., Peters, M., & Gardner, M. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.228 · Cited by 41
Collecting a large-scale gender bias dataset for coreference resolution and machine translation
Levy, S., Lazar, K., & Stanovsky, G. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.211 · Cited by 41
On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning
Jin, X., Barbieri, F., Kennedy, B., Mostafazadeh Davani, A., Neves, L., & Ren, X. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.296 · Cited by 40
Fine-Grained Human Feedback Gives Better Rewards for Language Model Training
Wu, Z., Hu, Y., Shi, W., Dziri, N., Suhr, A., Ammanabrolu, P., Smith, N. A., Ostendorf, M., & Hajishirzi, H. (2023) · arXiv · doi:10.48550/arxiv.2306.01693 · Cited by 40
Mishra, A., Patel, D., Vijayakumar, A., Li, X. L., Kapanipathi, P., & Talamadupula, K. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.104 · Cited by 39
Conceptor Debiasing of Word Representations Evaluated on WEAT
Karve, S., Ungar, L., & Sedoc, J. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3806 · Cited by 38
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
Manakul, P., Liusie, A., & Gales, M. J. F. (2023) · arXiv · doi:10.48550/arxiv.2303.08896 · Cited by 38
SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization
Laban, P., Schnabel, T., Bennett, P. N., & Hearst, M. A. (2021) · arXiv · doi:10.48550/arxiv.2111.09525 · Cited by 38
Do Language Models Know When They’re Hallucinating References?
Agrawal, A., Suzgun, M., Mackey, L., & Kalai, A. T. (2023) · arXiv · doi:10.48550/arxiv.2305.18248 · Cited by 37
Bitton, Y., Stanovsky, G., Schwartz, R., & Elhadad, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.9 · Cited by 36
Fairness in Language Models Beyond English: Gaps and Challenges
Ramesh, K., Sitaram, S., & Choudhury, M. (2023) · Findings of the Association for Computational Linguistics: EACL 2023 · doi:10.18653/v1/2023.findings-eacl.157 · Cited by 36
Vanmassenhove, E., Emmery, C., & Shterionov, D. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.704 · Cited by 35
Models See Hallucinations: Evaluating the Factuality in Video Captioning
Liu, H., & Wan, X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.723 · Cited by 35
Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models
Li, M., Peng, B., Galley, M., Gao, J., & Zhang, Z. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.12 · Cited by 35
Teaching Language Models to Hallucinate Less with Synthetic Tasks
Jones, E., Palangi, H., Simoes, C., Chandrasekaran, V., Mukherjee, S., Mitra, A., Awadallah, A. H., & Kamar, E. S. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.06827 · Cited by 35
Grep-BiasIR: A dataset for investigating gender representation bias in information retrieval results
Krieg, K., Parada-Cabaleiro, E., Medicus, G., Lesota, O., Schedl, M., & Rekabsaz, N. (2023) · Proceedings of the 2023 Conference on Human Information Interaction and Retrieval · doi:10.1145/3576840.3578295 · Cited by 31
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models
Zhao, J., Fang, M., Shi, Z., Li, Y., Chen, L., & Pechenizkiy, M. (2023) · arXiv · doi:10.48550/arxiv.2305.11262 · Cited by 31
Self-citationMonolingual and Multilingual Reduction of Gender Bias in Contextualized Representations
Liang, S., Dufter, P., & Schütze, H. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.446 · Cited by 29
KoLA: Carefully Benchmarking World Knowledge of Large Language Models
Yu, J., Wang, X., Tu, S., Cao, S., Zhang-Li, D., Lv, X., Peng, H., Yao, Z., Zhang, X., Li, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.09296 · Cited by 29
Chern, I.-C., Chern, S., Chen, S., Yuan, W., Feng, K., Zhou, C., He, J., Neubig, G., & Liu, P. (2023) · arXiv · doi:10.48550/arxiv.2307.13528 · Cited by 29
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
Chuang, Y.-S., Xie, Y., Luo, H., Kim, Y., Glass, J., & He, P. (2023) · arXiv · doi:10.48550/arxiv.2309.03883 · Cited by 29
Varshney, N., Yao, W., Zhang, H., Chen, J., & Yu, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.03987 · Cited by 28
Self-citation"Merge Conflicts!" Exploring the Impacts of External Distractors to Parametric Knowledge GraphsPreprint
Qian, C. S., Zhao, X., & Wu, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.08594 · Cited by 28
Improving Gender-Related Fairness in Sentence Encoders: A Semantics-Based Approach
Dolci, T., Azzalini, F., & Tanelli, M. (2023) · Data Science and Engineering · doi:10.1007/s41019-023-00211-0 · Cited by 23
Tool-Augmented Reward Modeling
Li, L., Chai, Y., Wang, S., Sun, Y., Tian, H., Zhang, N., & Wu, H. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.01045 · Cited by 22
Break, Perturb, Build: Automatic Perturbation of Reasoning Paths Through Question Decomposition
Geva, M., Wolfson, T., & Berant, J. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00450 · Cited by 21
The Role of Protected Class Word Lists in Bias Identification of Contextualized Word Representations
Sedoc, J., & Ungar, L. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3808 · Cited by 16
Using Natural Sentence Prompts for Understanding Biases in Language Models
Alnegheimish, S., Guo, A., & Sun, Y. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.203 · Cited by 16
Rawte, V., Priya, P., Tonmoy, S. M. T. I., Zaman, S. M. M., Sheth, A., & Das, A. (2023) · arXiv · doi:10.48550/arxiv.2309.11064 · Cited by 16
Easy Adaptation to Mitigate Gender Bias in Multilingual Text Classification
Huang, X. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.52 · Cited by 15
MEGA: Multilingual Evaluation of Generative AI
Ahuja, K., Diddee, H., Hada, R., Ochieng, M., Ramesh, K., Jain, P., Nambi, A., Ganu, T., Segal, S., Axmed, M., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12528 · Cited by 15
On measuring and mitigating biased inferences of word embeddings
Dev, S., Li, T., Phillips, J. M., & Srikumar, V. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6267 · Cited by 14
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models
Zhao, J., Fang, M., Shi, Z., Li, Y., Chen, L., & Pechenizkiy, M. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.757 · Cited by 14
Self-citationTransformer-Patcher: One Mistake worth One Neuron
Huang, Z., Shen, Y., Zhang, X., Zhou, J., Rong, W., & Xiong, Z. (2023) · arXiv · doi:10.48550/arxiv.2301.09785 · Cited by 14
(2023) · doi:10.48550/arxiv.2305.13669 · Cited by 13
Exposing Bias in Online Communities through Large-Scale Language ModelsPreprint
Wald, C., & Pfahler, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.02294 · Cited by 13
Fighting Bias With Bias: Promoting Model Robustness by Amplifying Dataset Biases
Reif, Y., & Schwartz, R. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.833 · Cited by 12
(2022) · doi:10.48550/arxiv.2210.08726 · Cited by 12
On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation
Wang, C., & Sennrich, R. (2020) · arXiv · doi:10.48550/arxiv.2005.03642 · Cited by 12
Chen, W., Wu, C.-K., Chen, H., & Chen, C.-C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.54 · Cited by 11
Wang, Z., Mao, S., Wu, W., Ge, T., Wei, F., & Ji, H. (2023) · arXiv · doi:10.48550/arxiv.2307.05300 · Cited by 11
Learning with Instance Bundles for Reading Comprehension
Dua, D., Dasigi, P., Singh, S., & Gardner, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.584 · Cited by 10
Generating benchmarks for factuality evaluation of language models
Muhlgay, D., Ram, O., Magar, I., Levine, Y., Ratner, N., Belinkov, Y., Abend, O., Leyton-Brown, K., Shashua, A., & Shoham, Y. (2023) · arXiv · doi:10.48550/arxiv.2307.06908 · Cited by 8
Human Feedback is not Gold Standard
Hosking, T., Blunsom, P., & Bartolo, M. (2023) · arXiv · doi:10.48550/arxiv.2309.16349 · Cited by 8
PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptions
Chen, A., Pasupat, P., Singh, S., Lee, H., & Guu, K. (2023) · arXiv · doi:10.48550/arxiv.2305.14908 · Cited by 7
FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation
Vu, T., Iyyer, M., Wang, X., Constant, N., Wei, J., Wei, J., Tar, C., Sung, Y.-H., Zhou, D., Le, Q., et al. (2023) · arXiv · Cited by 7
BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models
Cao, Q., Min, S., Wang, Y., & Hajishirzi, H. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.01329 · Cited by 6
Automatic and Human-AI Interactive Text GenerationPreprint
Dou, Y., Laban, P., Gardent, C., & Xu, W. M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.03878 · Cited by 6
FLEEK: Factual Error Detection and Correction with Evidence Retrieved from External KnowledgePreprint
Bayat, F. F., Qian, K., Han, B., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.17119 · Cited by 2
Instruction Mining: Instruction Data Selection for Tuning Large Language Models
Cao, Y., Kang, Y., Wang, C., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2307.06290 · Cited by 2
Sennrich, R., Vamvas, J., & Mohammadshahi, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.07098 · Cited by 2
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Kabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2023) · arXiv · doi:10.48550/arxiv.2308.02312
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Workshop, B., :, Scao, T. L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A. S., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.05100
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models
Maaz, M., Rasheed, H., Khan, S., & Khan, F. S. (2023) · arXiv · doi:10.48550/arxiv.2306.05424
Sinitsin, A., Plokhotnyuk, V., Pyrkin, D., Popov, S., & Babenko, A. (2020) · arXiv · doi:10.48550/arxiv.2004.00345
Research works printed but not linked: 30
- Banerjee S, Lavie A (2005) Meteor: an automatic metric for mt evaluation with improved correlation with human judgments. In: Proceedings of the ACL workshop on intrinsic and extrinsic evaluation meas- ures for machine translation and/or summarization, pp 65-72
- Bolukbasi T, Chang K-W, Zou JY, Saligrama V, Kalai AT (2016) Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Adv Neural Inf Process Syst 29
- Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877-1901
- Buolamwini J, Gebru T (2018) Gender shades: intersectional accuracy disparities in commercial gender classification. In: Conference on fairness, accountability and transparency. PMLR, pp 77-91
- Cao Y, Kang Y, Sun L (2023) Instruction mining: High-quality instruction data selection for large lan- guage models. arXiv preprint arXiv: 2307. 06290
- Carlini N, Tramer F, Wallace E, Jagielski M, Herbert-Voss A, Lee K, Roberts A, Brown T, Song D, Erlingsson U et al. (2021) Extracting training data from large language models. In: 30th USENIX security symposium (USENIX Security 21), pp 2633-2650
- Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, Barham P, Chung HW, Sutton C, Gehrmann S et al (2023) Palm: scaling language modeling with pathways. J Mach Learn Res 24(240):1-113
- Dev S, Phillips J (2019) Attenuating bias in word vectors. In: The 22nd international conference on arti- ficial intelligence and statistics. PMLR, pp 879-887
- Dinan E, Roller S, Shuster K, Fan A, Auli M, Weston J (2018) Wizard of Wikipedia: knowledge-pow- ered conversational agents. In: International conference on learning representations
- Gao L, Schulman J, Hilton J (2023) Scaling laws for reward model overoptimization. In: Krause A, Brunskill E, Cho K, Engelhardt B, Sabato S, Scarlett J (eds.), Proceedings of the 40th interna- tional conference on machine learning research, vol. 202. PMLR, pp 10835-10866. https:// proce edings. mlr. press/ v202/ gao23h. html
- Honovich O, Choshen L, Aharoni R, Neeman E, Szpektor I, Abend O (2021) Q2: evaluating factual consist- ency in knowledge-grounded dialogues via question generation and question answering. In: Proceed- ings of the 2021 conference on empirical methods in natural language processing, pp 7856-7870 · DOI · Paper page
- Kabir S, Udo-Imeh DN, Kou B, Zhang T (2023) Who answers it better? An in-depth analysis of chatgpt and stack overflow answers to software engineering questions. arXiv preprint arXiv: 2308. 02312
- Le Bras R, Swayamdipta S, Bhagavatula C, Zellers R, Peters M, Sabharwal A, Choi Y (2020) Adversarial filters of dataset biases. International conference on machine learning. PMLR, pp 1078-1088
- Lee N, Ping W, Xu P, Patwary M, Fung PN, Shoeybi M, Catanzaro B (2022) Factuality enhanced language models for open-ended text generation. Adv Neural Inf Process Syst 35:34586-34599
- Liu F, Lin K, Li L, Wang J, Yacoob Y, Wang L (2023) Aligning large multi-modal model with robust instruction tuning. arXiv preprint arXiv: 2306. 14565
- McFadden AC, Marsh GE, Price BJ, Hwang Y (1992) A study of race and gender bias in the punishment of school children. Educ Treat Child 15(2):140-146
- Meng K, Bau D, Andonian A, Belinkov Y (2022) Locating and editing factual associations in GPT. Adv Neural Inf Process Syst 35:17359-17372
- Ntoutsi E, Fafalios P, Gadiraju U, Iosifidis V, Nejdl W, Vidal M-E, Ruggieri S, Turini F, Papadopoulos S, Krasanakis E et al (2020) Bias in data-driven artificial intelligence systems-an introductory survey. Wiley Interdisc Rev: Data Min Knowl Discov 10(3):1356
- Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A et al (2022) Training language models to follow instructions with human feedback. Adv Neural Inf Process Syst 35:27730-27744
- Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I et al (2019) Language models are unsuper- vised multitask learners. OpenAI blog 1(8):9
- Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Explor- ing the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(1):5485-5551
- Sahlgren M, Olsson F (2019) Gender bias in pretrained Swedish embeddings. In: Proceedings of the 22nd Nordic conference on computational linguistics, pp 35-43
- Santhanam S, Hedayatnia B, Gella S, Padmakumar A, Kim S, Liu Y, Hakkani-Tür D (2021) Rome was built in 1776: a case study on factual correctness in knowledge-grounded response generation. In: EMNLP 2021 workshop on NLP for conversational AI
- Schmidt B (2015) Rejecting the gender binary: a vector-space operation. Ben’s Bookworm Blog
- Sun T, Zhang X, He Z, Li P, Cheng Q, Yan H, Liu X, Shao Y, Tang Q, Zhao X, et al. (2023) Moss: training conversational language models from synthetic data 7. arXiv preprint arXiv: 2307. 15020
- Tan YC, Celis LE (2019) Assessing social and intersectional biases in contextualized word representations. Adv Neural Inf Process Syst 32
- Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Atten- tion is all you need. Adv Neural Inf Process Syst 30
- Wiener N (1950) The human use of human beings: Cybernetics and society Workshop B, Scao TL, Fan A, Akiki C, Pavlick E, Ilić S, Hesslow D, Castagné R, Luccioni AS, Yvon F et al. (2022) Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv: 2211. 05100
- Zaheri S, Leath J, Stroud D (2020) Toxic comment classification. SMU Data Sci Rev 3(1):13
- Zhao T, Wei M, Preston JS, Poon H (2023) Automatic calibration and error correction for large language models via pareto optimal self-supervision. arXiv preprint arXiv: 2306. 16564
Cited by
Bias in Large Language Models: Origin, Evaluation, and Mitigation
Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2026) · Electronics · doi:10.3390/electronics15091824 · Cited by 249
“In alignment with causal methods in intra-model debiasing, researchers also develop methods based on causal prompting (Li et al., 2024a; Zhang et al., 2024).”
Bias in Large Language Models: Origin, Evaluation, and Mitigation
Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2024) · arXiv · doi:10.48550/arxiv.2411.10915 · Cited by 208
“In alignment with causal methods in intra-model debiasing, researchers also develop methods based on causal prompting (Li et al., 2024a; Zhang et al., 2024).”
A survey on multilingual large language models: corpora, alignment, and bias
Xu, Y., Hu, L., Zhao, J., Qiu, Z., Xu, K., Ye, Y., & Gu, H. (2025) · Frontiers of Computer Science · doi:10.1007/s11704-024-40579-4 · Cited by 225
Evaluating Trustworthiness in AI: Risks, Metrics, and Applications Across Industries
Nastoska, A., Jancheska, B., Rizinski, M., & Trajanov, D. (2025) · Electronics · doi:10.3390/electronics14132717 · Cited by 167
Abdel‐Rehim, A., Zenil, H., Orhobor, O. I., Fisher, M., Collins, R. J., Bourne, E., Fearnley, G. W., Tate, E., Smith, H. X., Soldatova, L., et al. (2025) · Journal of The Royal Society Interface · doi:10.1098/rsif.2024.0674 · Cited by 55
Metadata
- Authors
- Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H.
- Affiliations
- Beijing Institute of Technology (CN)
- Corresponding author
- Zhang, Huiyan
- Venue
- Artificial Intelligence Review
- Published
- 2024-08-10
- Volume
- 57
- Issue
- 9
- ISSN
- 0269-2821 (ISSN-L)
- 1573-7462
- OpenAlex
- W4401484139
- Cited by
- 141
- References
- 216
- Citations by year
- 2026: 70 · 2025: 69 · 2024: 2
- FWCI
- 58.44
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- Artificial Intelligence Review (publishedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, bias mitigation, hallucination mitigation, trustworthy AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.