CoChat
Sign Up
Cited by 43Topic ModelingAdvanced Text Analysis TechniquesSentiment Analysis and Opinion Mining

CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuning

Open access
Tang, Xiangru, Nair, Arjun, Wang, Borui, Wang, Bingyao, Desai, Jai, Wade, Aaron, Li, Haoran, Celikyilmaz, Asli +2 more·2022·Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
DOI 10.18653/v1/2022.naacl-main.415
Open this paper in CoChat — free accountOpen PDF

Abstract

Xiangru Tang, Arjun Nair, Borui Wang, Bingyao Wang, Jai Desai, Aaron Wade, Haoran Li, Asli Celikyilmaz, Yashar Mehdad, Dragomir Radev. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2022.naacl-main.415

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The CONFIT contrastive fine-tuning strategy can improve the factual consistency and overall quality of summaries in dialogue summarization.

    “Tang et al. [245] propose another contrastive fine-tuning strategy, named CONFIT, that can improve the factual consistency and overall quality of summaries.”

    From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [245]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 37

  1. Ziqiang Cao, Furu Wei, Wenjie Li, and Sujian Li. 2018. Faithful to the original: Fact aware neural abstrac- tive summarization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
  2. Jiaao Chen and Diyi Yang. 2020. Multi-view sequence- to-sequence models with conversational structure for abstractive dialogue summarization. In Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4106-4118, Online. Association for Computational Linguistics. · DOI
  3. Jiaao Chen and Diyi Yang. 2021a. Simple conver- sational data augmentation for semi-supervised ab- stractive dialogue summarization. In Proceedings of the Association for Computational Linguistics: EMNLP 2020, Online. Association for Computa- tional Linguistics.
  4. Jiaao Chen and Diyi Yang. 2021b. Structure-aware ab- stractive conversation summarization via discourse and action graphs. In Proceedings of the 2021 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies, pages 1380-1391, Online. As- sociation for Computational Linguistics. · DOI
  5. Yen-Chun Chen and Mohit Bansal. 2018. Fast abstrac- tive summarization with reinforce-selected sentence rewriting. In Proceedings of the 56th Annual Meet- ing of the Association for Computational Linguis- tics (Volume 1: Long Papers), pages 675-686, Mel- bourne, Australia. Association for Computational Linguistics. · DOI
  6. Yulong Chen, Yang Liu, Liang Chen, and Yue Zhang. 2021c. DialogSum: A real-life scenario dialogue summarization dataset. In Findings of the Associ- ation for Computational Linguistics: ACL-IJCNLP 2021, pages 5062-5074, Online. Association for Computational Linguistics. · DOI
  7. Yulong Chen, Yang Liu, and Yue Zhang. 2021d. Di- alogsum challenge: Summarizing real-life scenario dialogues. In Proceedings of the 14th International Conference on Natural Language Generation, pages 308-313.
  8. Alexander Fabbri, Faiaz Rahman, Imad Rizvi, Borui Wang, Haoran Li, Yashar Mehdad, and Dragomir Radev. 2021. ConvoSumm: Conversation summa- rization benchmark and improved abstractive sum- marization with argument mining. In Proceedings of the 59th Annual Meeting of the Association for Com- putational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6866-6880, Online. Association for Computational Linguistics. · DOI
  9. Xiachong Feng, Xiaocheng Feng, and Bing Qin. 2021a. A survey on dialogue summarization: Re- cent advances and new frontiers. arXiv preprint arXiv:2107.03175. · arXiv
  10. Xiachong Feng, Xiaocheng Feng, Libo Qin, Bing Qin, and Ting Liu. 2021b. Language model as an annota- tor: Exploring dialogpt for dialogue summarization. arXiv preprint arXiv:2105.12544. · arXiv
  11. Xiachong Feng, Xiaocheng Feng, Libo Qin, Bing Qin, and Ting Liu. 2021c. Language model as an an- notator: Exploring DialoGPT for dialogue summa- rization. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 11th International Joint Conference on Nat- ural Language Processing (Volume 1: Long Papers), pages 1479-1491, Online. Association for Computa- tional Linguistics. · DOI
  12. Beliz Gunel, Jingfei Du, Alexis Conneau, and Ves Stoy- anov. 2020. Supervised contrastive learning for pre- trained language model fine-tuning. arXiv preprint arXiv:2011.01403. · arXiv
  13. Iryna Gurevych and Michael Strube. 2004. Semantic similarity applied to spoken dialogue summarization. In COLING 2004: Proceedings of the 20th Inter- national Conference on Computational Linguistics, pages 764-770, Geneva, Switzerland. COLING.
  14. Jia Jin Koay, Alexander Roustai, Xiaojin Dai, Dillon Burns, Alec Kerrigan, and Fei Liu. 2020. How domain terminology affects meeting summarization performance. In Proceedings of the 28th Inter- national Conference on Computational Linguistics, pages 5689-5695, Barcelona, Spain (Online). Inter- national Committee on Computational Linguistics. · DOI
  15. Seolhwa Lee, Kisu Yang, Chanjun Park, João Sedoc, and Heuiseok Lim. 2021. Who says like a style of vitamin: Towards syntax-aware dialoguesumma- rization using multi-task learning. arXiv preprint arXiv:2109.14199. · arXiv
  16. Yuejie Lei, Yuanmeng Yan, Zhiyuan Zeng, Keqing He, Ximing Zhang, and Weiran Xu. 2021. Hier- archical speaker-aware sequence-to-sequence model for dialogue summarization. In ICASSP 2021- 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7823-7827. IEEE.
  17. Haoran Li, Junnan Zhu, Jiajun Zhang, and Chengqing Zong. 2018. Ensure the correctness of the summary: Incorporate entailment knowledge into abstractive sentence summarization. In Proceedings of the 27th International Conference on Computational Linguis- tics, pages 1430-1441, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  18. Chin-Yew Lin. 2004. ROUGE: A package for auto- matic evaluation of summaries. In Text Summariza- tion Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
  19. Junpeng Liu, Yanyan Zou, Hainan Zhang, Hongshen Chen, Zhuoye Ding, Caixia Yuan, and Xiaojie Wang. 2021a. Topic-aware contrastive learning for abstrac- tive dialogue summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natu- ral Language Processing (EMNLP), Online. Asso- ciation for Computational Linguistics.
  20. Zhengyuan Liu and Nancy F. Chen. 2021. Controllable neural dialogue summarization with personal named entity planning. In Proceedings of the Association for Computational Linguistics: EMNLP 2020, On- line. Association for Computational Linguistics.
  21. Zhengyuan Liu, Ke Shi, and Nancy Chen. 2021b. Coreference-aware dialogue summarization. In Pro- ceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 509-519, Singapore and Online. Association for Computational Linguistics.
  22. Iain McCowan, Jean Carletta, Wessel Kraaij, Simone Ashby, S Bourban, M Flynn, M Guillemot, Thomas Hain, J Kadlec, Vasilis Karaiskos, et al. 2005. The ami meeting corpus. In Proceedings of the 5th inter- national conference on methods and techniques in behavioral research, volume 88, page 100.
  23. Rada Mihalcea and Paul Tarau. 2004. TextRank: Bringing order into text. In Proc. of EMNLP, pages 404-411, Barcelona, Spain. Association for Compu- tational Linguistics.
  24. Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to- text transformer. Journal of Machine Learning Re- search, 21(140):1-67.
  25. Clément Rebuffel, Thomas Scialom, Laure Soulier, Benjamin Piwowarski, Sylvain Lamprier, Jacopo Staiano, Geoffrey Scoutheeten, and Patrick Galli- nari. 2021. Data-questeval: A referenceless metric for data to text semantic evaluation. arXiv preprint arXiv:2104.07555. · arXiv
  26. Harvey Sacks, Emanuel A Schegloff, and Gail Jeffer- son. 1978. A simplest systematics for the organiza- tion of turn taking for conversation. In Studies in the organization of conversational interaction, pages 7- 55. Elsevier.
  27. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer- generator networks. In Proc. of ACL, pages 1073- 1083, Vancouver, Canada. Association for Computa- tional Linguistics. · DOI · Paper page
  28. Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020. Asking and answering questions to evaluate the fac- tual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Com- putational Linguistics, pages 5008-5020, Online. Association for Computational Linguistics. · DOI
  29. Chien-Sheng Wu, Linqing Liu, Wenhao Liu, Pontus Stenetorp, and Caiming Xiong. 2021. Controllable abstractive dialogue summarization with sketch su- pervision. In Findings of the Association for Com- putational Linguistics: ACL-IJCNLP 2021, pages 5108-5122, Online. Association for Computational Linguistics. · DOI
  30. Feng Xiachong, Feng Xiaocheng, and Qin Bing. 2021. Incorporating commonsense knowledge into ab- stractive dialogue summarization via heterogeneous graph networks. In Proceedings of the 20th Chinese National Conference on Computational Linguistics, pages 964-975, Huhhot, China. Chinese Informa- tion Processing Society of China.
  31. Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. Bartscore: Evaluating generated text as text genera- tion. arXiv preprint arXiv:2106.11520. · arXiv
  32. Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Pe- ter Liu. 2020. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In In- ternational Conference on Machine Learning, pages 11328-11339.
  33. PMLR.
  34. Shiyue Zhang, Asli Celikyilmaz, Jianfeng Gao, and Mohit Bansal. 2021. EmailSum: Abstractive email thread summarization. In Proceedings of the 59th Annual Meeting of the Association for Computa- tional Linguistics and the 11th International Joint Conference on Natural Language Processing (Vol- ume 1: Long Papers), pages 6895-6909, Online. As- sociation for Computational Linguistics. · DOI
  35. Lulu Zhao, Weiran Xu, and Jun Guo. 2020. Improving abstractive dialogue summarization with graph struc- tures and topic words. In Proceedings of the 28th International Conference on Computational Linguis- tics, pages 437-449, Barcelona, Spain (Online). In- ternational Committee on Computational Linguis- tics. · DOI
  36. Chenguang Zhu, Yang Liu, Jie Mei, and Michael Zeng. 2021. Mediasum: A large-scale media interview dataset for dialogue summarization. arXiv preprint arXiv:2103.06410. · arXiv
  37. Chenguang Zhu, Ruochen Xu, Michael Zeng, and Xue- dong Huang. 2020. A hierarchical network for ab- stractive meeting summarization with cross-domain pretraining. In Findings of the Association for Com- putational Linguistics: EMNLP 2020, pages 194- 203, Online. Association for Computational Linguis- tics. · DOI

Cited by

Sorted by citations

Authors
Tang, X., Nair, A., Wang, B., Wang, B., Desai, J., Wade, A., Li, H., Celikyilmaz, A., Mehdad, Y., & Radev, D.
Affiliations
Yale University (US)
ORCID
Venue
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Publisher
Association for Computational Linguistics
Published
2022
DOI
10.18653/v1/2022.naacl-main.415
OpenAlex
W4287887652
Cited by
43
References
6
Citations by year
2025: 4 · 2024: 8 · 2023: 24 · 2022: 7
FWCI
5.94
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Keywords
dialogue summarization, contrastive fine-tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review