CoChat
Sign Up
PreprintCited by 1Topic ModelingAdversarial Robustness in Machine LearningNatural Language Processing Techniques

Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Models

Open access
Arteaga, Gabriel Y., Schön, Thomas B., Pielawski, Nicolas·2024·arXiv
DOI 10.48550/arxiv.2409.02976arXiv 2409.02976
Open this paper in CoChat — free accountOpen PDF

Abstract

Uncertainty estimation is a necessary component when implementing AI in high-risk settings, such as autonomous cars, medicine, or insurances. Large Language Models (LLMs) have seen a surge in popularity in recent years, but they are subject to hallucinations, which may cause serious harm in high-risk settings. Despite their success, LLMs are expensive to train and run: they need a large amount of computations and memory, preventing the use of ensembling methods in practice. In this work, we present a novel method that allows for fast and memory-friendly training of LLM ensembles. We show that the resulting ensembles can detect hallucinations and are a viable approach in practice as only one GPU is needed for training and inference.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2409.02976

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Fine-tuned LLMs exhibit reduced hallucination in text generation compared to their general-purpose counterparts.

    “Studies have revealed that fine-tuned LLMs exhibit reduced hallucination in text generation compared to their general-purpose counterparts (Azaria and Mitchell, 2023; Xiong et al., 2024; Arteaga et al., 2024).”

    From MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models (2025) · cites this paper as Arteaga et al., 2024

    The Introduction and Conclusion describe fine-tuning models to detect hallucinations, not reducing hallucinations in the text those models generate. The Models and Baselines section compares fine-tuned approaches but does not report a hallucination-rate comparison with general-purpose models. I searched for that comparison and found no passage addressing it. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 45

  1. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, P. Schuh, K. Shi, S. Tsvyashchenko, J. Maynez, A. Rao, P. Barnes, Y. Tay, N. Shazeer, V. Prabhakaran, E. Reif, N. Du, B. Hutchinson, R. Pope, J. Bradbury, J. Austin, M. Isard, G. Gur-Ari, P. Yin, T. Duke, A. Levskaya, S. Ghemawat, S. Dev, H. Michalewski, X. Garcia, V. Misra, K. Robinson, L. Fedus, D. Zhou, D. Ippolito, D. Luan, H. Lim, B. Zoph, A. Spiri- donov, R. Sepassi, D. Dohan, S. Agrawal, M. Omernick, A. M. Dai, T. S. Pillai, M. Pellat, A. Lewkowycz, E. Moreira, R. Child, O. Polo- zov, K. Lee, Z. Zhou, X. Wang, B. Saeta, M. Diaz, O. Firat, M. Catasta, J. Wei, K. Meier- Hellstern, D. Eck, J. Dean, S. Petrov, and N. Fiedel. "PaLM: Scaling Language Modeling with Pathways". In: Journal of Machine Learn- ing Research 24.240 (2023), pp. 1-113. issn: 1533-7928. url: http://jmlr.org/papers/ v24/22-1144.html (visited on 02/22/2024).
  2. L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. Liu. A Survey on Hallucina- tion in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. arXiv:2311.05232 [cs]. Nov. 2023. doi: 10 . 48550 / arXiv . 2311 . 05232. url: http : / / arxiv . org / abs / 2311 . 05232 (visited on 02/22/2024). · DOI · Paper page
  3. W. Kryscinski, B. McCann, C. Xiong, and R. Socher. "Evaluating the Factual Consis- tency of Abstractive Text Summarization". In: Proceedings of the 2020 Conference on Empir- ical Methods in Natural Language Processing (EMNLP). Ed. by B. Webber, T. Cohn, Y. He, and Y. Liu. Online: Association for Com- putational Linguistics, Nov. 2020, pp. 9332- 9346. doi: 10.18653/v1/2020.emnlp-main. 750. url: https://aclanthology.org/2020. emnlp-main.750 (visited on 06/11/2024). · DOI · Paper page
  4. O. Honovich, L. Choshen, R. Aharoni, E. Neeman, I. Szpektor, and O. Abend. "Q 2 : Evaluating Factual Consistency in Knowledge- Grounded Dialogues via Question Generation and Question Answering". In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Ed. by M.-F. Moens, X. Huang, L. Specia, and S. W.-t. Yih. Online and Punta Cana, Dominican Repub- lic: Association for Computational Linguistics, Nov. 2021, pp. 7856-7870. doi: 10 . 18653 / v1/2021.emnlp-main.619. url: https:// aclanthology.org/2021.emnlp-main.619 (visited on 04/18/2024). · DOI · Paper page
  5. P. Laban, T. Schnabel, P. N. Bennett, and M. A. Hearst. "SummaC: Re-Visiting NLI- based Models for Inconsistency Detection in Summarization". In: Transactions of the As- sociation for Computational Linguistics 10 (2022).
  6. Ed. by B. Roark and A. Nenkova. Place: Cambridge, MA Publisher: MIT Press, pp. 163-177. doi: 10.1162/tacl_a_00453. url: https : / / aclanthology . org / 2022 . tacl-1.10 (visited on 06/12/2024). · DOI · Paper page
  7. A. Malinin and M. Gales. "Uncertainty Esti- mation in Autoregressive Structured Predic- tion". In: (2020). Publisher: arXiv Version Number: 5. doi: 10 . 48550 / ARXIV . 2002 . 07650. url: https://arxiv.org/abs/2002. 07650 (visited on 02/22/2024). · DOI · Paper page
  8. Y. Xiao and W. Y. Wang. "On Hallucina- tion and Predictive Uncertainty in Condi- tional Language Generation". In: (2021). Pub- lisher: arXiv Version Number: 1. doi: 10 . 48550/ARXIV.2103. 15025. url: https:// arxiv . org / abs / 2103 . 15025 (visited on 02/22/2024). · DOI
  9. A. Gleave and G. Irving. Uncertainty Estimation for Language Reward Models. arXiv:2203.07472 [cs]. 2022. doi: 10.48550/ ARXIV . 2203 . 07472. url: http : / / arxiv . org / abs / 2203 . 07472 (visited on 06/11/2024). · DOI · Paper page
  10. M. Sun, W. Yan, P. Abbeel, and I. Mor- datch. "Quantifying Uncertainty in Founda- tion Models via Ensembles". In: NeurIPS 2022 Workshop on Robustness in Sequence Model- ing. Nov. 2022. url: https://openreview. net / forum ? id = LpBlkATV24M (visited on 04/24/2024).
  11. B. Hou, Y. Liu, K. Qian, J. Andreas, S. Chang, and Y. Zhang. Decomposing Uncertainty for Large Language Models through Input Clarifi- cation Ensembling. arXiv:2311.08718 [cs]. Nov. 2023. doi: 10 . 48550 / arXiv . 2311 . 08718. url: http://arxiv.org/abs/2311.08718 (visited on 04/22/2024). · DOI
  12. Y. Huang, J. Song, Z. Wang, S. Zhao, H. Chen, F. Juefei-Xu, and L. Ma. "Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models". In: (2023). Publisher: arXiv Version Number: 3. doi: 10.48550/ARXIV.2307.10236. url: https://arxiv.org/abs/2307.10236 (vis- ited on 02/22/2024). · DOI
  13. L. Kuhn, Y. Gal, and S. Farquhar. "Semantic Uncertainty: Linguistic Invariances for Uncer- tainty Estimation in Natural Language Gen- eration". In: (2023). Publisher: arXiv Version Number: 3. doi: 10 . 48550 / ARXIV . 2302 . 09664. url: https://arxiv.org/abs/2302. 09664 (visited on 02/22/2024). · DOI · Paper page
  14. Z. Lin, S. Trivedi, and J. Sun. Generat- ing with Confidence: Uncertainty Quantifi- cation for Black-box Large Language Models. arXiv:2305.19187 [cs, stat]. Oct. 2023. doi: 10 . 48550 / arXiv . 2305 . 19187. url: http : //arxiv.org/abs/2305.19187 (visited on 04/22/2024). · DOI
  15. C. Chen, K. Liu, Z. Chen, Y. Gu, Y. Wu, M. Tao, Z. Fu, and J. Ye. "INSIDE: LLMs' Inter- nal States Retain the Power of Hallucination Detection". In: (2024). Publisher: arXiv Ver- sion Number: 1. doi: 10.48550/ARXIV.2402. 03744. url: https://arxiv.org/abs/2402. 03744 (visited on 02/22/2024). · DOI
  16. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. "Lora: Low-rank adaptation of large language models". In: arXiv preprint arXiv:2106.09685 (2021). · arXiv
  17. J. Maynez, S. Narayan, B. Bohnet, and R. Mc- Donald. "On Faithfulness and Factuality in Abstractive Summarization". In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Ed. by D. Ju- rafsky, J. Chai, N. Schluter, and J. Tetreault. Online: Association for Computational Lin- guistics, July 2020, pp. 1906-1919. doi: 10. 18653/v1/2020.acl-main.173. url: https: //aclanthology.org/2020.acl-main.173 (visited on 08/22/2024). · DOI · Paper page
  18. M. Cao, Y. Dong, and J. Cheung. "Halluci- nated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization". In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Ed. by S. Mure- san, P. Nakov, and A. Villavicencio. Dublin, Ireland: Association for Computational Lin- guistics, May 2022, pp. 3340-3354. doi: 10. 18653/v1/2022.acl-long.236. url: https: //aclanthology.org/2022.acl-long.236 (visited on 04/18/2024). · DOI · Paper page
  19. S. Huo, N. Arabzadeh, and C. L. A. Clarke. Retrieving Supporting Evidence for LLMs Gen- erated Answers. arXiv:2306.13781 [cs]. June 2023. doi: 10 . 48550 / arXiv . 2306 . 13781. url: http://arxiv.org/abs/2306.13781 (visited on 04/19/2024). · DOI · Paper page
  20. B. Lattimer, P. CHen, X. Zhang, and Y. Yang. "Fast and Accurate Factual Inconsistency De- tection Over Long Documents". In: Proceed- ings of the 2023 Conference on Empirical Methods in Natural Language Processing. Ed.
  21. by H. Bouamor, J. Pino, and K. Bali. Singa- pore: Association for Computational Linguis- tics, Dec. 2023, pp. 1691-1703. doi: 10.18653/ v1/2023.emnlp-main.105. url: https:// aclanthology.org/2023.emnlp-main.105 (visited on 06/11/2024). · DOI
  22. S. Min, K. Krishna, X. Lyu, M. Lewis, W.-t. Yih, P. Koh, M. Iyyer, L. Zettlemoyer, and H. Hajishirzi. "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation". In: Proceedings of the 2023 Conference on Empirical Meth- ods in Natural Language Processing. Ed. by H. Bouamor, J. Pino, and K. Bali. Singapore: As- sociation for Computational Linguistics, Dec. 2023, pp. 12076-12100. doi: 10.18653/v1/ 2023 . emnlp -main . 741. url: https : / / aclanthology.org/2023.emnlp-main.741 (visited on 06/11/2024). · DOI · Paper page
  23. J. Chen, G. Kim, A. Sriram, G. Durrett, and E. Choi. Complex Claim Verification with Evi- dence Retrieved in the Wild. arXiv:2305.11859 [cs]. May 2023. doi: 10.48550/arXiv.2305. 11859. url: http://arxiv.org/abs/2305. 11859 (visited on 04/18/2024). · DOI
  24. I.-C. Chern, S. Chern, S. Chen, W. Yuan, K. Feng, C. Zhou, J. He, G. Neubig, and P. Liu. FacTool: Factuality Detection in Gen- erative AI -A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios. arXiv:2307.13528 [cs]. July 2023. doi: 10 . 48550 / arXiv . 2307 . 13528. url: http : / / arxiv . org / abs / 2307 . 13528 (visited on 04/19/2024). · DOI · Paper page
  25. R. Cohen, M. Hamri, M. Geva, and A. Glober- son. "LM vs LM: Detecting Factual Errors via Cross Examination". In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Ed. by H. Bouamor, J. Pino, and K. Bali. Singapore: As- sociation for Computational Linguistics, Dec. 2023, pp. 12621-12640. doi: 10.18653/v1/ 2023 . emnlp -main . 778. url: https : / / aclanthology.org/2023.emnlp-main.778 (visited on 06/12/2024). · DOI · Paper page
  26. B. Lakshminarayanan, A. Pritzel, and C. Blundell. "Simple and Scalable Predictive Un- certainty Estimation using Deep Ensembles". In: Advances in Neural Information Process- ing Systems. Vol. 30. Curran Associates, Inc., 8 2017. url: https://proceedings.neurips. cc / paper _ files / paper / 2017 / hash / 9ef2ed4b7fd2c810847ffa5fa85bce38 - Abstract.html (visited on 02/23/2024).
  27. A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed. Mistral 7B. arXiv:2310.06825 [cs]. Oct. 2023. doi: 10.48550/arXiv.2310. 06825. url: http://arxiv.org/abs/2310. 06825 (visited on 02/22/2024). · DOI · Paper page
  28. A. Dubey et al. The Llama 3 Herd of Mod- els. 2024. arXiv: 2407.21783 [cs.AI]. url: https://arxiv.org/abs/2407.21783. · arXiv
  29. Y. Wen, D. Tran, and J. Ba. "BatchEnsem- ble: an Alternative Approach to Efficient En- semble and Lifelong Learning". In: Interna- tional Conference on Learning Representa- tions. Sept. 2019. url: https://openreview. net / forum ? id = Sklf1yrYDr (visited on 04/10/2024).
  30. D. Tran, J. Z. Liu, M. W. Dusenberry, D. Phan, M. Collier, J. Ren, K. Han, Z. Wang, Z. E. Ma- riet, H. Hu, N. Band, T. G. J. Rudner, K. Sing- hal, Z. Nado, J. v. Amersfoort, A. Kirsch, R. Jenatton, N. Thain, H. Yuan, E. K. Buchanan, K. P. Murphy, D. Sculley, Y. Gal, Z. Ghahra- mani, J. Snoek, and B. Lakshminarayanan. "Plex: Towards Reliability using Pretrained Large Model Extensions". In: First Work- shop on Pre-training: Perspectives, Pitfalls, and Paths Forward at ICML 2022. July 2022. url: https://openreview.net/forum?id= 6x0gB9gOHFg (visited on 06/12/2024).
  31. X. Wang, L. Aitchison, and M. Rudolph. LoRA ensembles for large language model fine-tuning. arXiv:2310.00035 [cs]. Oct. 2023. doi: 10 . 48550 / arXiv . 2310 . 00035. url: http : / / arxiv . org / abs / 2310 . 00035 (visited on 04/22/2024). · DOI
  32. Y. Zhai, H. Zhang, Y. Lei, Y. Yu, K. Xu, D. Feng, B. Ding, and H. Wang. "Uncertainty- penalized reinforcement learning from human feedback with diverse reward lora ensembles". In: arXiv preprint arXiv:2401.00243 (2023). · arXiv
  33. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. "LoRA: Low-Rank Adaptation of Large Lan- guage Models". In: International Conference on Learning Representations. Oct. 2021. url: https : / / openreview . net / forum ? id = nZeVKeeFYf9 (visited on 06/12/2024).
  34. O. Balabanov and H. Linander. Uncertainty quantification in fine-tuned LLMs using LoRA ensembles. arXiv:2402.12264 [cs, stat]. 2024. doi: 10 . 48550 / ARXIV . 2402 . 12264. url: http : / / arxiv . org / abs / 2402 . 12264 (vis- ited on 06/11/2024). · DOI
  35. E. Hüllermeier and W. Waegeman. "Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods". en. In: Machine Learning 110.3 (Mar. 2021), pp. 457-506. issn: 1573-0565. doi: 10.1007/ s10994-021-05946-3. url: https://doi. org/10.1007/s10994-021-05946-3 (visited on 04/29/2024). · DOI · Paper page
  36. T. Pearce, F. Leibfried, and A. Brintrup. "Un- certainty in Neural Networks: Approximately Bayesian Ensembling". In: Proceedings of the Twenty Third International Conference on Ar- tificial Intelligence and Statistics. ISSN: 2640- 3498. PMLR, June 2020, pp. 234-244. url: https : / / proceedings . mlr . press / v108 / pearce20a.html (visited on 02/23/2024).
  37. P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang. "SQuAD: 100,000+ Questions for Ma- chine Comprehension of Text". In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Ed. by J. Su, K. Duh, and X. Carreras. Austin, Texas: As- sociation for Computational Linguistics, Nov. 2016, pp. 2383-2392. doi: 10.18653/v1/D16- 1264. url: https : / / aclanthology . org / D16-1264 (visited on 06/12/2024). · DOI · Paper page
  38. P. Rajpurkar, R. Jia, and P. Liang. "Know What You Don't Know: Unanswerable Ques- tions for SQuAD". In: Proceedings of the 56th Annual Meeting of the Association for Com- putational Linguistics (Volume 2: Short Pa- pers). Ed. by I. Gurevych and Y. Miyao. Mel- bourne, Australia: Association for Computa- tional Linguistics, July 2018, pp. 784-789. doi: 10.18653/v1/P18-2124. url: https: //aclanthology.org/P18-2124 (visited on 06/12/2024). · DOI · Paper page
  39. D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. "Mea- suring Massive Multitask Language Under- standing". In: International Conference on Learning Representations. 2021. url: https: //openreview.net/forum?id=d7KBjmI3GmQ.
  40. M. Zhang, O. Press, W. Merrill, A. Liu, and N. A. Smith. "How Language Model Hallu- cinations Can Snowball". In: (2023). Pub- lisher: arXiv Version Number: 1. doi: 10 . 48550/ARXIV.2305. 13534. url: https:// arxiv . org / abs / 2305 . 13534 (visited on 02/22/2024). · DOI · Paper page
  41. A. Azaria and T. Mitchell. "The Internal State of an LLM Knows When It's Lying". In: Find- ings of the Association for Computational Lin- guistics: EMNLP 2023.
  42. Ed. by H. Bouamor, J. Pino, and K. Bali. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 967- 976. doi: 10 . 18653 / v1 / 2023 . findings - emnlp . 68. url: https : / / aclanthology . org/2023.findings-emnlp.68 (visited on 06/12/2024). · DOI · Paper page
  43. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blon- del, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. "Scikit- learn: Machine Learning in Python". In: Jour- nal of Machine Learning Research 12 (2011), pp. 2825-2830.
  44. K. He, X. Zhang, S. Ren, and J. Sun. "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification". In: Proceedings of the IEEE International Confer- ence on Computer Vision. 2015, pp. 1026-1034. url: https : / / openaccess . thecvf . com / content _ iccv _ 2015 / html / He _ Delving _ Deep_into_ICCV_2015_paper.html (visited on 05/05/2024).
  45. X. Glorot and Y. Bengio. "Understanding the difficulty of training deep feedforward neu- ral networks". In: Proceedings of the Thir- teenth International Conference on Artificial Intelligence and Statistics. ISSN: 1938-7228. JMLR Workshop and Conference Proceed- ings, Mar. 2010, pp. 249-256. url: https : //proceedings.mlr.press/v9/glorot10a. html (visited on 05/05/2024).

Cited by

Sorted by citations

Authors
Arteaga, G. Y., Schön, T. B., & Pielawski, N.
Affiliations
Uppsala University (SE)
ORCID
Pielawski, Nicolas
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2409.02976
arXiv
2409.02976
OpenAlex
W6982168548
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • Diva portal (Dalarna University Library) (submittedVersion, other-oa)
  • arXiv (Cornell University) (cc-by)
Funders
  • Knut och Alice Wallenbergs Stiftelse
  • Uppsala Universitet
  • HORIZON EUROPE European Research Council (101054643)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
hallucination detection, large language models, LLM ensembles, uncertainty estimation, memory-efficient training, fine-tuned models, GPU training, high-risk AI systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review