Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models
Open accessThe DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Large language models (LLMs) have gained widespread adoption in various natural language processing tasks, including question answering and dialogue systems. However, a major drawback of LLMs is the issue of hallucination, where they generate unfaithful or inconsistent content that deviates from the input source, leading to severe consequences. In this paper, we propose a robust discriminator named RelD to effectively detect hallucination in LLMs' generated answers. RelD is trained on the constructed RelQA, a bilingual question-answering dialogue dataset along with answers generated by LLMs and a comprehensive set of metrics. Our experimental results demonstrate that the proposed RelD successfully detects hallucination in the answers generated by diverse LLMs. Moreover, it performs well in distinguishing hallucination in LLMs' generated answers from both in-distribution and out-of-distribution datasets. Additionally, we also conduct a thorough analysis of the types of hallucinations that occur and present valuable insights. This research significantly contributes to the detection of reliable answers generated by LLMs and holds noteworthy implications for mitigating hallucination in the future work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 1×
“RelQA consists of nine sub-datasets: SQuAD [55], DuReader [24], HotpotQA[75],MSMARCO[46],NewsQA [69],QuAC[11],CoQA[58], TriviaQA-Web [29], and TriviaQA-Wikipedia [29].”
MethodSurvey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Chen, D., Dai, W., et al. (2022) · arXiv · doi:10.48550/arxiv.2202.03629 · Cited by 4,981
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342 · cited 1×
“…metrics [21, 66, 72], model-based metrics (including Information Extraction (IE)-based metric, QA-based metric [25, 57, 60], Natural Language Inference (NLI) Metrics [26, 32, 73], Faithfulness Classification Metrics [25, 41, 79], LM-based Metrics [19, 67]), and human-based evaluations [61, 65].”
Background
Cited by
"I look at it as the king of knowledge": How Blind People Use and Understand Generative AI Tools
Adnin, R., & Das, M. (2024) · The 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675631 · Cited by 73
Shorinwa, O., Mei, Z., Lidard, J., Ren, A. Z., & Majumdar, A. (2026) · ACM Computing Surveys · doi:10.1145/3744238 · Cited by 61
Automating the Development of Task-oriented LLM-based Chatbots
Sánchez Cuadrado, J., Pérez-Soler, S., Guerra, E., & De Lara, J. (2024) · ACM Conversational User Interfaces 2024 · doi:10.1145/3640794.3665538 · Cited by 46
Zhao, Q., Wang, R., Cen, Y., Zha, D., Tan, S., Dong, Y., & Tang, J. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.1259 · Cited by 32
Sood, A. K., Zeadally, S., & Hong, E. (2025) · Computers and Electrical Engineering · doi:10.1016/j.compeleceng.2025.110307 · Cited by 33
Developing a Framework for Auditing Large Language Models using Human-in-the-loop
Amirizaniani, M., Lavergne, A., Snell Okada, E., Chadha, A., Roosta, T., & Shah, C. (2025) · Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region · doi:10.1145/3767695.3769514 · Cited by 16
Perera, M., Ananthanarayan, S., Goncu, C., & Marriott, K. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713634 · Cited by 14
Tang, Z., Wang, R., Chen, W., Zheng, Y., Chen, Z., Liu, Y., Wang, K., Chen, T., & Lin, L. (2023) · arXiv · doi:10.48550/arxiv.2308.11914 · Cited by 4
Multi-Agent Large Language Models for Conversational Task-Solving
Becker, J. (2024) · arXiv · doi:10.48550/arxiv.2410.22932 · Cited by 3
Metadata
- Authors
- Chen, Y., Fu, Q., Yuan, Y., Wen, Z., Fan, G., Liu, D., Zhang, D., Li, Z., & Xiao, Y.
- Venue
- Proceedings of the 32nd ACM International Conference on Information and Knowledge Management
- Publisher
- ACM
- Published
- 2023-10-21
- Pages
- 245-255
- OpenAlex
- W4387848774
- Cited by
- 94
- References
- 48
- FWCI
- 17.77
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.