The DOI matches its DataCite record.
RAGged Edges: The Double-Edged Sword of Retrieval-Augmented Chatbots
Open accessAbstract
Large language models (LLMs) like ChatGPT demonstrate the remarkable progress of artificial intelligence. However, their tendency to hallucinate -- generate plausible but false information -- poses a significant challenge. This issue is critical, as seen in recent court cases where ChatGPT's use led to citations of non-existent legal rulings. This paper explores how Retrieval-Augmented Generation (RAG) can counter hallucinations by integrating external knowledge with prompts. We empirically evaluate RAG against standard LLMs using prompts designed to induce hallucinations. Our results show that RAG increases accuracy in some cases, but can still be misled when prompts directly contradict the model's pre-trained understanding. These findings highlight the complex nature of hallucinations and the need for more robust solutions to ensure LLM reliability in real-world applications. We offer practical recommendations for RAG deployment and discuss implications for the development of more trustworthy LLMs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Context engineering delivers substantial performance improvements through techniques like retrievalaugmented generation and superposition prompting, achieving documented improvements including 18-fold enhancement in text navigation accuracy, 94% success rates, and significant gains from careful prompt construction and automatic optimization across specialized domains [271, 774, 687].”
From A Survey of Context Engineering for Large Language Models (2025) · cites this paper as [271]
In the citing paper
“Context engineering delivers substantial performance improvements through techniques like retrievalaugmented generation and superposition prompting, achieving documented improvements including 18-fold enhancement in text navigation accuracy, 94% success rates, and significant gains from careful prompt construction and automatic optimization across specialized domains [271, 774, 687].”
From A Survey of Context Engineering for Large Language Models (2025) · cites this paper as [271]
In the citing paper
“Context engineering delivers substantial performance improvements through techniques like retrievalaugmented generation and superposition prompting, achieving documented improvements including 18-fold enhancement in text navigation accuracy, 94% success rates, and significant gains from careful prompt construction and automatic optimization across specialized domains [271, 774, 687].”
From A Survey of Context Engineering for Large Language Models (2025) · cites this paper as [271]
In the citing paper
“Context engineering delivers substantial performance improvements through techniques like retrievalaugmented generation and superposition prompting, achieving documented improvements including 18-fold enhancement in text navigation accuracy, 94% success rates, and significant gains from careful prompt construction and automatic optimization across specialized domains [271, 774, 687].”
From A Survey of Context Engineering for Large Language Models (2025) · cites this paper as [271]
Citations
Cited by
Sorted by citations
A Survey of Context Engineering for Large Language Models
Mei, L., Yao, J., Ge, Y., Wang, Y., Bi, B., Cai, Y., Liu, J., Li, M., Li, Z.-Z., Zhang, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.13334 · Cited by 281
Metadata
- Authors
- Feldman, P., Foulds, J. R., & Pan, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2403.01193
- OpenAlex
- W6922271154
- Cited by
- 30
- References
- 0
- Citations by year
- 2026: 25 · 2025: 2 · 2024: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- Maryland Shared Open Access Repository (USMAI Consortium) (submittedVersion)
- arXiv (Cornell University)
- Digital Repository at the University of Maryland (University of Maryland College Park)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- retrieval-augmented generation, hallucinations, large language models, ChatGPT, prompt engineering, external knowledge integration, LLM reliability, legal citation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
