Surveyor: A System for Generating Coherent Survey Articles for Scientific Topics
Open accessAbstract
We investigate the task of generating coherent survey articles for scientific topics. We introduce an extractive summarization algorithm that combines a content model with a discourse model to generate coherent and readable summaries of scientific topics using text from scientific articles relevant to the topic. Human evaluation on 15 topics in computational linguistics shows that our system produces significantly more coherent summaries than previous systems. Specifically, our system improves the ratings for coherence by 36% in human evaluation compared to C-Lexrank, a state of the art system for scientific article summarization.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 25
- Barzilay, R., and Lee, L. 2004. Catching the drift: Probabilistic content models, with applications to generation and summarization. In Susan Dumais, D. M., and Roukos, S., eds., HLT-NAACL 2004: Main Proceedings , 113-120. Boston, Massachusetts, USA: Association for Computational Linguistics.
- Barzilay, R., and McKeown, K. R. 2005. Sentence fusion for multidocument news summarization. Computational Linguistics 31(3):297-328.
- Bollegala, D.; Okazaki, N.; and Ishizuka, M. 2010. A bottomup approach to sentence ordering for multi-document summarization. Information Processing & Management 46(1):89 109.
- Christensen, J.; Mausam; Soderland, S.; and Etzioni, O. 2013. Towards coherent multi-document summarization. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2013) .
- Daum´ e, III, H., and Marcu, D. 2006. Bayesian query-focused summarization. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics , ACL-44, 305-312. Stroudsburg, PA, USA: Association for Computational Linguistics.
- de Marneffe, M.-C.; MacCartney, B.; and Manning, C. D. 2006. Generating typed dependency parses from phrase structure parses. In The International Conference on Language Resources and Evaluation (LREC) , volume 6, 449-454. Citeseer.
- De Meo, P.; Ferrara, E.; Fiumara, G.; and Provetti, A. 2011. Generalized louvain method for community detection in large networks. CoRR abs/1108.1502. · arXiv
- Elhadad, N., and McKeown, K. R. 2001. Towards generating patient specific summaries of medical articles. In In Proceedings of NAACL-2001 Automatic Summarization Workshop .
- Erkan, G., and Radev, D. R. 2004. Lexrank: Graph-based centrality as salience in text summarization. Journal of Artificial Intelligence Research (JAIR) .
- Fried, D., and Kobourov, S. G. 2013. Maps of Computer Science. ArXiv e-prints .
- Fung, P., and Ngai, G. 2006. One story, one flow: Hidden markov story models for multilingual multidocument summarization. ACM Trans. Speech Lang. Process. 3(2):1-16.
- Haghighi, A., and Vanderwende, L. 2009. Exploring content models for multi-document summarization. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics , NAACL '09, 362-370. Stroudsburg, PA, USA: Association for Computational Linguistics.
- Jha, R.; Abu-Jbara, A.; and Radev, D. R. 2013. A system for summarizing scientific topics starting from keywords. In Proceedings of The Association for Computational Linguistics (short paper) .
- Kan, M.; McKeown, K. R.; and Klavans, J. L. 2001. Domainspecific informative and indicative summarization for information retrieval. In Proc. of the Document Understanding Conference (DUC , 6.
- Lin, Z.; Liu, C.; Ng, H. T.; and Kan, M.-Y. 2012. Combining coherence models and machine translation evaluation metrics for summarization evaluation. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1 , ACL '12, 1006-1014. Stroudsburg, PA, USA: Association for Computational Linguistics.
- Lin, C. Y. 2004a. Looking for a few good metrics: Automatic summarization evaluation - how many samples are enough? In Proceedings of the NTCIR Workshop 4 .
- Lin, C.-Y. 2004b. ROUGE: A package for automatic evaluation of summaries. In Proceedings of the ACL workshop on Text Summarization Branches Out .
- Mohammad, S.; Dorr, B.; Egan, M.; Hassan, A.; Muthukrishan, P.; Qazvinian, V.; Radev, D.; and Zajic, D. 2009. Using citations to generate surveys of scientific paradigms. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics , NAACL '09, 584-592. Stroudsburg, PA, USA: Association for Computational Linguistics.
- Nanba, H.; Kando, N.; and Okumura, M. 2004. Classification of research papers using citation links and citation types: Towards automatic review article generation. In Proceedings of the 11th SIG Classification Research Workshop , 117-134.
- Nenkova, A., and McKeown, K. 2011. Automatic summarization. Foundations and Trends in Information Retrieval 5(23):103-233.
- Norman, G. 2010. Likert scales, levels of measurement and the laws of statistics. Advances in Health Sciences Education 15(5):625-632.
- Prasad, R.; Dinesh, N.; Lee, A.; Miltsakaki, E.; Robaldo, L.; Joshi, A.; and Webber, B. 2008. The penn discourse treebank 2.0. In In Proceedings of LREC .
- Qazvinian, V., and Radev, D. R. 2008. Scientific paper summarization using citation summary networks. In COLING 2008 .
- Shahaf, D.; Guestrin, C.; and Horvitz, E. 2012. Metro maps of science. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , KDD '12, 1122-1130. New York, NY, USA: ACM.
- Yoo, I.; Hu, X.; and Song, I.-Y. 2006. A coherent biomedical literature clustering and summarization approach through ontology-enriched graphical representations. In Tjoa, A., and Trujillo, J., eds., Data Warehousing and Knowledge Discovery , volume 4081 of Lecture Notes in Computer Science . Springer Berlin Heidelberg. 374-383.
Cited by
Sorted by citations
TLDR: Extreme Summarization of Scientific Documents
Cachola, I., Lo, K., Cohan, A., & Weld, D. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.428 · Cited by 169 · see citing works
“, 2020), automated survey generation (Mohammad et al., 2009; Jha et al., 2015; Fabbri et al., 2018; Wang et al., 2018), and other techniques focusing on exploiting the unique properties of scientific documents such as long length and structure (Conroy and Davis, 2017; Nikolov et al.”
BackgroundCan We Automate Scientific Reviewing?
Yuan, W., Liu, P., & Neubig, G. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12862 · Cited by 96 · see citing works
Chern, I.-C., Chern, S., Chen, S., Yuan, W., Feng, K., Zhou, C., He, J., Neubig, G., & Liu, P. (2023) · arXiv · doi:10.48550/arxiv.2307.13528 · Cited by 29 · see citing works
“Scientific Literature Review Writing The scientific literature review writing task (Jha et al., 2015) aims to analyze and synthesize existing research on a specific topic in a field of study.”
Background
- Authors
- Jha, R., Coke, R., & Radev, D.
- Affiliations
- University of Michigan (US)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2015-02-19
- Volume
- 29
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2163104394
- Cited by
- 19
- References
- 0
- Citations by year
- 2024: 2 · 2023: 1 · 2022: 3 · 2021: 1 · 2020: 1 · 2019: 3 · 2017: 1 · 2016: 3 · 2015: 4
- FWCI
- 1.50
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Text Readability and Simplification
- Keywords
- extractive summarization, coherence, discourse model, content model, computational linguistics, human evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
