De-Hallucinator: Mitigating LLM Hallucinations in Code Generation Tasks via Iterative Grounding
The DOI matches its DataCite record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Large language models (LLMs) trained on datasets of publicly available source code have established a new state of the art in code generation tasks. However, these models are mostly unaware of the code that exists within a specific project, preventing the models from making good use of existing APIs. Instead, LLMs often invent, or "hallucinate", non-existent APIs or produce variants of already existing code. This paper presents De-Hallucinator, a technique that grounds the predictions of an LLM through a novel combination of retrieving suitable API references and iteratively querying the model with increasingly suitable context information in the prompt. The approach exploits the observation that predictions by LLMs often resemble the desired code, but they fail to correctly refer to already existing APIs. De-Hallucinator automatically identifies project-specific API references related to the model's initial predictions and adds these references into the prompt. Unlike retrieval-augmented generation (RAG), our approach uses the initial prediction(s) by the model to iteratively retrieve increasingly suitable API references. Our evaluation applies the approach to two tasks: predicting API usages in Python and generating tests in JavaScript. We show that De-Hallucinator consistently improves the generated code across five LLMs. In particular, the approach improves the edit distance by 23.3-50.6% and the recall of correctly predicted API usages by 23.9-61.0% for code completion, and improves the number of fixed tests that initially failed because of hallucinations by 63.2%, resulting in a 15.5% increase in statement coverage for test generation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“De-Hallucinator [241] retrieves API references using first-time generated contents, then conducts query-based RAG for improved code completion.”
From Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024) · cites this paper as [241]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Precise Zero-Shot Dense Retrieval without Relevance Labels
Gao, L., Ma, X., Lin, J., & Callan, J. (2022) · arXiv · doi:10.48550/arxiv.2212.10496 · Cited by 362
StarCoder: may the source be with you!
Li, R., Allal, L. B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.06161 · Cited by 261
Less training, more repairing please: revisiting automated program repair via zero-shot learning
Xia, C. S., & Zhang, L. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549101 · Cited by 231
A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges
Liang, J. T., Yang, C., & Myers, B. A. (2023) · arXiv · doi:10.48550/arxiv.2303.17125 · Cited by 206
Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproduction
Kang, S., Yoon, J., & Yoo, S. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00194 · Cited by 204
Impact of Code Language Models on Automated Program Repair
Jiang, N., Liu, K., Lutellier, T., & Tan, L. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00125 · Cited by 202
Pythia: AI-assisted Code Completion System
Svyatkovskiy, A., Zhao, Y., Fu, S., & Sundaresan, N. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330699 · Cited by 169
CoCoMIC: Code Completion by Jointly Modeling In-file and Cross-file Context
Ding, Y., Wang, Z., Ahmad, W. U., Ramanathan, M. K., Nallapati, R., Bhatia, P., Roth, D., & Xiang, B. (2022) · arXiv · doi:10.48550/arxiv.2212.10007 · Cited by 20
Cited by
Retrieval-Augmented Generation for AI-Generated Content: A Survey
Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2024) · arXiv · doi:10.48550/arxiv.2402.19473 · Cited by 240
“It filters out lower-quality data to maintain a high standard of information mapping, offering a variety of retrieval system options like TF-IDF, DPR, RAG, and BLINK + flair to support evidence-based predictions or citations according to task requirements.”
(2025) · doi:10.1109/icse55347.2025.00157 · Cited by 157
You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projects
Bouzenia, I., & Pradel, M. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3728922 · Cited by 35
“To make code generation practical also for real-world software development, researchers have proposed techniques to augment prompts with repository-level context [7, 9, 30, 44].”
Self-citation
Metadata
- Authors
- Eghbali, A., & Pradel, M.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- arXiv
- 2401.01701
- References
- 9
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: De-Hallucinator: Mitigating LLM Hallucinations in Code Generation Tasks via Iterative Grounding. https://app.cochat.ai/papers/10.48550/arxiv.2401.01701
- The paper
- Eghbali, A., & Pradel, M. (2024). De-Hallucinator: Mitigating LLM Hallucinations in Code Generation Tasks via Iterative Grounding. arXiv. https://doi.org/10.48550/arxiv.2401.01701
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.