CoChat

Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization

Chrysostomou, George, Zhao, Zhixue, Williams, Miles, Aletras, Nikolaos · 2024 · Transactions of the Association for Computational Linguistics
Cited by 18

The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.

Abstract

Despite the remarkable performance of generative large language models (LLMs) on abstractive summarization, they face two significant challenges: their considerable size and tendency to hallucinate. Hallucinations are concerning because they erode reliability and raise safety issues. Pruning is a technique that reduces model size by removing redundant weights, enabling more efficient sparse inference. Pruned models yield downstream task performance comparable to the original, making them ideal alternatives when operating on a limited budget. However, the effect that pruning has upon hallucinations in abstractive summarization with LLMs has yet to be explored. In this paper, we provide an extensive empirical study across five summarization datasets, two state-of-the-art pruning methods, and five instruction-tuned LLMs. Surprisingly, we find that hallucinations are less prevalent from pruned LLMs than the original models. Our analysis suggests that pruned models tend to depend more on the source document for summary generation. This leads to a higher lexical overlap between the generated summary and the source document, which could be a reason for the reduction in hallucination risk.1

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1162/tacl_a_00695

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 49

  1. Adams, 2023, From sparse to dense: GPT-4 summarization with chain of density prompting, 68, 10.18653/v1/2023.newsum-1.7 · DOI
  2. Almazrouei, 2023, The Falcon series of open language models, arXiv preprint, arXiv:2311.16867 · arXiv
  3. Cao, 2018, Faithful to the original: Fact-aware neural abstractive summarization, 10.1609/aaai.v32i1.11912 · DOI · Paper page
  4. Choubey, 2023, CaPE: Contrastive parameter ensembling for reducing hallucination in abstractive summarization, 10755, 10.18653/v1/2023.findings-acl.685 · DOI
  5. Chrysostomou, 2022, An empirical study on explanations in out-of-domain settings, 6920, 10.18653/v1/2022.acl-long.477 · DOI
  6. Ciosici, 2021, Perhaps PTLMs should go to school – a task to assess open book and closed book QA, 6104, 10.18653/v1/2021.emnlp-main.493 · DOI
  7. Cohn, 2008, Sentence compression beyond word deletion, 137, 10.3115/1599081.1599099 · DOI · Paper page
  8. Dettmers, 2022, GPT3.int8(): 8-bit matrix multiplication for transformers at scale
  9. Dun, 2023, Sweeping heterogeneity with smart mops: Mixture of prompts for LLM task adaptation, arXiv preprint
  10. Fierro, 2022, Factual consistency of multilingual pretrained language models, 3046, 10.18653/v1/2022.findings-acl.240 · DOI
  11. Frantar, 2022, Optimal brain compression: A framework for accurate post-training quantization and pruning, 4475
  12. Frantar, 2023, OPTQ: Accurate quantization for generative pre-trained transformers
  13. Guerreiro, 2023, Optimal transport for unsupervised hallucination detection in neural machine translation, 13766, 10.18653/v1/2023.acl-long.770 · DOI
  14. Guzmán, 2019, The FLORES evaluation datasets for low-resource machine translation: Nepali–English and Sinhala–English, 6098, 10.18653/v1/D19-1632 · DOI · Paper page
  15. Hagiwara, 1994, A simple and effective method for removal of hidden units and weights, Neurocomputing, 6, 207, 10.1016/0925-2312(94)90055-8 · DOI
  16. Han, 2015, Learning both weights and connections for efficient neural network
  17. Hasan, 2024, Pruning for protection: Increasing jailbreak resistance in aligned LLMs without fine-tuning, arXiv preprint
  18. Hassibi, 1993, Optimal brain surgeon and general network pruning, 293, 10.1109/ICNN.1993.298572 · DOI · Paper page
  19. Huang, 2020, What have we achieved on text summarization?, 446, 10.18653/v1/2020.emnlp-main.33 · DOI · Paper page
  20. Srinivasan Iyer , XiVictoria Lin, RamakanthPasunuru, TodorMihaylov, DanielSimig, PingYu, KurtShuster, TianluWang, QingLiu, Punit SinghKoura, XianLi, BrianO’Horo, GabrielPereyra, JeffWang, ChristopherDewan, AsliCelikyilmaz, LukeZettlemoyer, and VesStoyanov. 2023. OPT-IML: Scaling language model instruction meta learning through the lens of generalization. arXiv preprint, arXiv:2212.12017. · arXiv
  21. Jaiswal, 2024, Compressing LLMs: The truth is rarely pure and never simple
  22. Ji, 2023, Towards mitigating LLM hallucination via self reflection, 1827, 10.18653/v1/2023.findings-emnlp.123 · DOI · Paper page
  23. Jiang, 2023, Mistral 7B, arXiv preprint
  24. King, 2022, Don’t say what you don’t know: Improving the consistency of abstractive summarization by constraining beam search, 555, 10.18653/v1/2022.gem-1.51 · DOI
  25. Laban, 2023, SummEdits: Measuring LLM ability at factual reasoning through the lens of summarization, 9662, 10.18653/v1/2023.emnlp-main.600 · DOI
  26. Lango, 2023, Critic-driven decoding for mitigating hallucinations in data-to-text generation, 2853, 10.18653/v1/2023.emnlp-main.172 · DOI
  27. Md, 2023, Building real-world meeting summarization systems using large language models: A practical perspective, 343, 10.18653/v1/2023.emnlp-industry.33 · DOI
  28. LeCun, 1989, Optimal brain damage
  29. Lin, 2004, ROUGE: A package for automatic evaluation of summaries, 74
  30. Ma, 2023, LLM-Pruner: On the structural pruning of large language models
  31. Manor, 2019, Plain English summarization of contracts, 1, 10.18653/v1/W19-2201 · DOI
  32. Merity, 2017, Pointer sentinel mixture models
  33. Milintsevich, 2023, Calvados at MEDIQA-chat 2023: Improving clinical note generation with multi-task instruction finetuning, 529, 10.18653/v1/2023.clinicalnlp-1.56 · DOI
  34. Namburi, 2023, The cost of compression: Investigating the impact of compression on parametric knowledge in language models, 5255, 10.18653/v1/2023.findings-emnlp.349 · DOI
  35. OpenAI, JoshAchiam, StevenAdler, SandhiniAgarwal, LamaAhmad, IlgeAkkaya, Florencia LeoniAleman, DiogoAlmeida, JankoAltenschmidt, SamAltman, ShyamalAnadkat, RedAvila, IgorBabuschkin, SuchirBalaji, ValerieBalcom, PaulBaltescu, HaimingBao, MohammadBavarian, JeffBelgum, IrwanBello, JakeBerdine, GabrielBernadett-Shapiro, ChristopherBerner, LennyBogdonoff, OlegBoiko, MadelaineBoyd, Anna-LuisaBrakman, GregBrockman, TimBrooks, MilesBrundage, KevinButton, TrevorCai, RosieCampbell, AndrewCann, BrittanyCarey, ChelseaCarlson, RoryCarmichael, BrookeChan, CheChang, FotisChantzis, DerekChen, SullyChen, RubyChen, JasonChen, MarkChen, BenChess, ChesterCho, CaseyChu, Hyung WonChung, DaveCummings, JeremiahCurrier, YunxingDai, CoryDecareaux, ThomasDegry, NoahDeutsch, DamienDeville, ArkaDhar, DavidDohan, SteveDowling, SheilaDunning, AdrienEcoffet, AttyEleti, TynaEloundou, DavidFarhi, LiamFedus, NikoFelix, Simón PosadaFishman, JustonForte, IsabellaFulford, LeoGao, ElieGeorges, ChristianGibson, VikGoel, TarunGogineni, GabrielGoh, RaphaGontijo-Lopes, JonathanGordon, MorganGrafstein, ScottGray, RyanGreene, JoshuaGross, Shixiang ShaneGu, YufeiGuo, ChrisHallacy, JesseHan, JeffHarris, YuchenHe, MikeHeaton, JohannesHeidecke, ChrisHesse, AlanHickey, WadeHickey, PeterHoeschele, BrandonHoughton, KennyHsu, ShengliHu, XinHu, JoostHuizinga, ShantanuJain, ShawnJain, JoanneJang, AngelaJiang, RogerJiang, HaozhunJin, DennyJin, ShinoJomoto, BillieJonn, HeewooJun, TomerKaftan, ŁukaszKaiser, AliKamali, IngmarKanitscheider, Nitish ShirishKeskar, TabarakKhan, LoganKilpatrick, Jong WookKim, ChristinaKim, YongjikKim, Jan HendrikKirchner, JamieKiros, MattKnight, DanielKokotajlo, ŁukaszKondraciuk, AndrewKondrich, ArisKonstantinidis, KyleKosic, GretchenKrueger, VishalKuo, MichaelLampe, IkaiLan, TeddyLee, JanLeike, JadeLeung, DanielLevy, Chak MingLi, RachelLim, MollyLin, StephanieLin, MateuszLitwin, TheresaLopez, RyanLowe, PatriciaLue, AnnaMakanju, KimMalfacini, SamManning, TodorMarkov, YanivMarkovski, BiancaMartin, KatieMayer, AndrewMayne, BobMcGrew, Scott MayerMcKinney, ChristineMcLeavey, PaulMcMillan, JakeMcNeil, DavidMedina, AalokMehta, JacobMenick, LukeMetz, AndreyMishchenko, PamelaMishkin, VinnieMonaco, EvanMorikawa, DanielMossing, TongMu, MiraMurati, OlegMurk, DavidMély, AshvinNair, ReiichiroNakano, RajeevNayak, ArvindNeelakantan, RichardNgo, HyeonwooNoh, LongOuyang, CullenO’Keefe, JakubPachocki, AlexPaino, JoePalermo, AshleyPantuliano, GiambattistaParascandolo, JoelParish, EmyParparita, AlexPassos, MikhailPavlov, AndrewPeng, AdamPerelman, Filipede Avila Belbute Peres, MichaelPetrov, Henrique Pondede Oliveira Pinto, MichaelPokorny, MichellePokrass, Vitchyr H.Pong, TollyPowell, AletheaPower, BorisPower, ElizabethProehl, RaulPuri, AlecRadford, JackRae, AdityaRamesh, CameronRaymond, FrancisReal, KendraRimbach, CarlRoss, BobRotsted, HenriRoussez, NickRyder, MarioSaltarelli, TedSanders, ShibaniSanturkar, GirishSastry, HeatherSchmidt, DavidSchnurr, JohnSchulman, DanielSelsam, KylaSheppard, TokiSherbakov, JessicaShieh, SarahShoker, PranavShyam, SzymonSidor, EricSigler, MaddieSimens, JordanSitkin, KatarinaSlama, IanSohl, BenjaminSokolowsky, YangSong, NatalieStaudacher, Felipe PetroskiSuch, NatalieSummers, IlyaSutskever, JieTang, NikolasTezak, Madeleine B.Thompson, PhilTillet, AminTootoonchian, ElizabethTseng, PrestonTuggle, NickTurley, JerryTworek, Juan Felipe CerónUribe, AndreaVallone, ArunVijayvergiya, ChelseaVoss, CarrollWainwright, Justin JayWang, AlvinWang, BenWang, JonathanWard, JasonWei, C. J.Weinmann, AkilaWelihinda, PeterWelinder, JiayiWeng, LilianWeng, MattWiethoff, DaveWillner, ClemensWinter, SamuelWolrich, HannahWong, LaurenWorkman, SherwinWu, JeffWu, MichaelWu, KaiXiao, TaoXu, SarahYoo, KevinYu, QimingYuan, WojciechZaremba, RowanZellers, ChongZhang, MarvinZhang, ShengjiaZhao, TianhaoZheng, JuntangZhuang, WilliamZhuk, and BarretZoph. 2024. GPT-4 technical report. arXiv preprint, arXiv:2303.08774. · arXiv
  36. Saggion, 2013, Automatic Text Summarization: Past, Present and Future, 3, 10.1007/978-3-642-28569-1_1 · DOI · Paper page
  37. Son, 2022, HaRiM +: Evaluating summary quality with hallucination risk, 895
  38. Tang, 2023, Understanding factual errors in summarization: Errors, summarizers, datasets, error detectors, 11626, 10.18653/v1/2023.acl-long.650 · DOI
  39. Tang, 2023, In-context learning of large language models for controlled dialogue summarization: A holistic benchmark and empirical analysis, 56, 10.18653/v1/2023.newsum-1.6 · DOI
  40. Touvron, 2023, Llama 2: Open foundation and fine-tuned chat models, arXiv preprint
  41. Wallace, 2021, Generating (factual?) narrative summaries of RCTs: Experiments with neural multi-document summarization, AMIA Summits on Translational Science Proceedings, 2021, 605
  42. Williams, 2023, How does calibration data affect the post-training pruning and quantization of large language models?, arXiv preprint
  43. Wolf, 2020, Transformers: State-of-the-art natural language processing, 38, 10.18653/v1/2020.emnlp-demos.6 · DOI · Paper page
  44. Guangxuan, 2022, Can model compression improve NLP fairness, arXiv preprint, arXiv:2201.08542 · arXiv
  45. Weijia, 2023, Understanding and detecting hallucinations in neural machine translation via model introspection, Transactions of the Association for Computational Linguistics, 11, 546, 10.1162/tacl_a_00563 · DOI · Paper page
  46. Zhao, 2023, Incorporating attribution importance for improving faithfulness metrics, 4732, 10.18653/v1/2023.acl-long.261 · DOI
  47. Zhao, 2022, On the impact of temporal concept drift on model explanations, 4039, 10.18653/v1/2022.findings-emnlp.298 · DOI
  48. Zhao, 2024, Reagent: A model-agnostic feature attribution method for generative language models, arXiv preprint, arXiv:2402.00794, 10.22541/au.170709121.16176681/v1 · DOI
  49. Zhao, 2022, Utilizing subjectivity level to mitigate identity term bias in toxic comments classification, Online Social Networks and Media, 29, 100205, 10.1016/j.osnem.2022.100205 · DOI

Cited by

See all 18

Metadata

Authors
Chrysostomou, G., Zhao, Z., Williams, M., & Aletras, N.
Affiliations
  • AstraZeneca, UK
  • University of Sheffield, UK. zhixue.zhao@sheffield.ac.uk
  • University of Sheffield, UK. mwilliams15@sheffield.ac.uk
  • University of Sheffield, UK. n.aletras@sheffield.ac.uk
Venue
Transactions of the Association for Computational Linguistics
Publisher
MIT Press
Published
2024
Volume
12
Pages
1163-1181
ISSN
2307-387X
Cited by
18
References
42
Language
en

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.