Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
PreprintCited by 1

The DOI matches its DataCite record; 5 of the 5 references it cites were found in a registry or an index.

Text Summarization with Latent Queries

Xu, Yumo, Lapata, Mirella·2021·arXiv
DOI 10.48550/arxiv.2106.00104arXiv 2106.00104
Open this paper in CoChat — free account

Abstract

The availability of large-scale datasets has driven the development of neural models that create summaries from single documents, for generic purposes. When using a summarization system, users often have specific intents with various language realizations, which, depending on the information need, can range from a single keyword to a long narrative composed of multiple questions. Existing summarization systems, however, often either fail to support or act robustly on this query focused summarization task. We introduce LaQSum, the first unified text summarization system that learns Latent Queries from documents for abstractive summarization with any existing query forms. Under a deep generative framework, our system jointly optimizes a latent query model and a conditional language model, allowing users to plug-and-play queries of any type at test time. Despite learning from only generic summarization data and requiring no further optimization for downstream summarization tasks, our system robustly outperforms strong comparison systems across summarization benchmarks with different query types, document settings, and target domains.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2106.00104

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 5 references

Research works printed but not linked: 43

  1. Deen Mohammad Abdullah and Yllias Chali. 2020. To- wards generating query to perform query focused ab- stractive summarization using pre-trained model. In Proceedings of the 13th International Conference on Natural Language Generation, pages 80-85, Dublin, Ireland.
  2. Rama Badrinath, Suresh Venkatasubramaniyan, and CE Veni Madhavan. 2011. Improving query fo- cused summarization using look-ahead strategy. In Proceedings of the 33rd European Conference on Advances in Information Retrieval, pages 641-652, Dublin, Ireland.
  3. Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. 2016. MS MARCO: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268. · arXiv
  4. Tal Baumel, Raphael Cohen, and Michael Elhadad. 2016. Topic concentration in query focused summa- rization datasets. In Proceedings of the 30th AAAI Conference on Artificial Intelligence, pages 2573- 2579, Phoenix, Arizona.
  5. Tal Baumel, Matan Eyal, and Michael Elhadad. 2018. Query focused abstractive summarization: Incorpo- rating query relevance, multi-document coverage, and summary length constraints into seq2seq mod- els. arXiv preprint arXiv:1801.07704. · arXiv
  6. Souradip Chakraborty, Ekaba Bisong, Shweta Bhatt, Thomas Wagner, Riley Elliott, and Francesco Mosconi. 2020. BioMedBERT: A pre-trained biomedical language model for qa and ir. In Pro- ceedings of the 28th International Conference on Computational Linguistics, pages 669-679, Online.
  7. Hoa Trang Dang. 2005. Overview of duc 2005. In Proceedings of the 2005 Document Understanding Conference, pages 1-12, Vancouver, Canada.
  8. Hoa Trang Dang. 2006. DUC 2005: Evaluation of question-focused summarization systems. In Pro- ceedings of the Workshop on Task-Focused Sum- marization and Question Answering, pages 48-55, Stroudsburg, PA, USA.
  9. William B Dolan and Chris Brockett. 2005. Automati- cally constructing a corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing, pages 9-16, Jeju Island, Korea.
  10. Zi-Yi Dou, Pengfei Liu, Hiroaki Hayashi, Zhengbao Jiang, and Graham Neubig. 2020. GSum: A general framework for guided neural abstractive summariza- tion. arXiv preprint arXiv:2010.08014. · arXiv
  11. Sebastian Gehrmann, Yuntian Deng, and Alexander Rush. 2018. Bottom-up abstractive summarization. In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 4098-4109, Brussels, Belgium.
  12. Karl Moritz Hermann, Tomáš Kočiský, Edward Grefen- stette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Proceedings of the 28th In- ternational Conference on Neural Information Pro- cessing Systems, page 1693-1701, Cambridge, MA, USA.
  13. Higgins, Loïc Matthey, A. Pal, Christopher P. Burgess, Xavier Glorot, M. Botvinick, S. Mohamed, and Alexander Lerchner. 2017. beta-vae: Learning basic visual concepts with a constrained variational framework. In ICLR.
  14. TD Hoa. 2006. Overview of duc 2006. In Proceed- ings of the 2006 Document Understanding Confer- ence, New York, USA.
  15. Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categor- ical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144. · arXiv
  16. HW Kuhn, AW Tucker, et al. 1951. Nonlinear program- ming. In Proceedings of the Second Berkeley Sym- posium on Mathematical Statistics and Probability.
  17. Md Tahmid Rahman Laskar, Enamul Hoque, and Jimmy Huang. 2020a. Query focused abstrac- tive summarization via incorporating query rele- vance and transfer learning with transformer models. In Canadian Conference on Artificial Intelligence, pages 342-348. Springer.
  18. Md Tahmid Rahman Laskar, Enamul Hoque, and Jimmy Xiangji Huang. 2020b. WSL-DS: Weakly su- pervised learning with distant supervision for query focused multi-document abstractive summarization. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5647-5654, Online.
  19. Logan Lebanoff, Kaiqiang Song, and Fei Liu. 2018. Adapting the neural encoder-decoder framework from single to multi-document summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4131-4141, Brussels, Belgium.
  20. Piji Li, Wai Lam, Lidong Bing, Weiwei Guo, and Hang Li. 2017a. Cascaded attention based unsupervised information distillation for compressive summariza- tion. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2081-2090, Brussells, Belgium.
  21. Piji Li, Zihao Wang, Wai Lam, Zhaochun Ren, and Lidong Bing. 2017b. Salience estimation via varia- tional auto-encoders for multi-document summariza- tion. In Proceedings of the 31th AAAI Conference on Artificial Intelligence, pages 3497-3503, San Fran- cisco, California, USA.
  22. Yang Liu and Mirella Lapata. 2019a. Hierarchical transformers for multi-document summarization. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 5070- 5081, Florence, Italy.
  23. Yang Liu and Mirella Lapata. 2019b. Text summariza- tion with pretrained encoders. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China.
  24. Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar Gulcehre, and Bing Xiang. 2016. Abstrac- tive text summarization using sequence-to-sequence RNNs and beyond. In Proceedings of The 20th SIGNLL Conference on Computational Natural Lan- guage Learning, pages 280-290, Berlin, Germany.
  25. Preksha Nema, Mitesh M. Khapra, Anirban Laha, and Balaraman Ravindran. 2017. Diversity driven at- tention model for query-based abstractive summa- rization. In Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguistics, pages 1063-1072, Vancouver, Canada.
  26. Laura Perez-Beltrachini, Yang Liu, and Mirella Lap- ata. 2019. Generating summaries with topic tem- plates and structured convolutional decoders. In Pro- ceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 5107- 5116, Florence, Italy.
  27. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Nat- ural Language Processing, pages 2383-2392, Syd- ney, Australia.
  28. Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sen- tence summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Lan- guage Processing, pages 379-389, Lisbon, Portugal.
  29. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer- generator networks. In Proceedings of the 55th An- nual Meeting of the Association for Computational Linguistics, pages 1073-1083, Vancouver, Canada.
  30. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 1715-1725, Berlin, Germany.
  31. Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber. 2015. Training very deep networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 2, pages 2377-2385, Montreal, Quebec, Canada.
  32. Dan Su, Yan Xu, Genta Indra Winata, Peng Xu, Hyeondey Kim, Zihan Liu, and Pascale Fung. 2019. Generalizing question answering system with pre- trained language model fine-tuning. In Proceedings of the 2nd Workshop on Machine Reading for Ques- tion Answering, pages 203-211, Hong Kong, China.
  33. Dan Su, Yan Xu, Tiezheng Yu, Farhad Bin Siddique, Elham Barezi, and Pascale Fung. 2020. CAiRE- COVID: A question answering and query-focused multi-document summarization system for COVID- 19 scholarly information management. In Proceed- ings of the 1st Workshop on NLP for COVID-19 at EMNLP 2020, Online.
  34. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems, pages 6000-6010.
  35. Xiaojun Wan. 2008. Using only cross-document rela- tionships for both generic and topic-focused multi- document summarizations. Information Retrieval, 11(1):25-49.
  36. Xiaojun Wan, Jianwu Yang, and Jianguo Xiao. 2007. Manifold-ranking based topic-focused multi- document summarization. In Proceedings of the 20th International Joint Conference on Artificial In- telligence, pages 2903-2908, Hyderabad, India.
  37. Xiaojun Wan and Jianmin Zhang. 2014. CTSUM: ex- tracting more certain summaries for news articles. In Proceedings of the 37th international ACM SIGIR Conference on Research & Development in Informa- tion Retrieval, pages 787-796, New York, United States.
  38. Zhengjue Wang, Zhibin Duan, Hao Zhang, Chaojie Wang, Long Tian, Bo Chen, and Mingyuan Zhou. 2020. Friendly topic assistant for transformer based abstractive summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 485-497, Online.
  39. Yumo Xu and Mirella Lapata. 2020a. Abstractive query focused summarization with query-free re- sources. arXiv preprint arXiv:2012.14774. · arXiv
  40. Yumo Xu and Mirella Lapata. 2020b. Coarse-to-fine query focused multi-document summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3632-3645, Online.
  41. Ming Zhong, Pengfei Liu, Yiran Chen, Danqing Wang, Xipeng Qiu, and Xuanjing Huang. 2020. Extractive summarization as text matching. In Proceedings of the 58th Annual Meeting of the Association for Com- putational Linguistics, pages 6197-6208, Online.
  42. Chenguang Zhu, William Hinthorn, Ruochen Xu, Qingkai Zeng, Michael Zeng, Xuedong Huang, and Meng Jiang. 2021. Enhancing factual consistency of abstractive summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 718-733, On- line.
  43. Haichao Zhu, Li Dong, Furu Wei, Bing Qin, and Ting Liu. 2019. Transforming wikipedia into augmented data for query-focused summarization. arXiv preprint arXiv:1911.03324. · arXiv

Cited by

Metadata

Authors
Xu, Y., & Lapata, M.
Venue
arXiv
Publisher
arXiv
Published
2021
DOI
10.48550/arxiv.2106.00104
arXiv
2106.00104
References
5
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.