Automatic Related Work Section in Scientific Article: Research Trends and Future Directions
The DOI matches its Crossref record; 39 of the 40 references it cites were found in a registry or an index.
Abstract
Related work plays an important role in the scientific article to track the history of topic evolution and demonstrate the improvement of the proposed method. Writing related work tasks could be arduous for young researchers and scholars. Some studies have attempted to design systems to ease this work besides improving the quality of related work produced. Initially, the research focused on extractive approaches by exploiting the heuristic method, machine learning, or graph-based methods. Recently, deep learning and its variants have become more popular because they can handle abstractive approaches. We propose five research questions that direct discussion and analysis of research in related work summarization. We look for answers to this research question in scientific articles on ScienceDirect, Web of Science, or the Google Scholar Database. This research presents a widespread survey about the trends in related work tasks: related work structure, important aspects, scientific corpora standards, prior proposed methods, baseline systems comparison, evaluation metrics, and challenges in the future. The results of this study provide an overview of research trends in related work summarization, as well as describe research gaps and future research that can still be developed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Historically, this stage did not benefit from the use of artificial intelligence techniques (Justitia and Wang 2022; Li and Ouyang 2022).”
From Artificial intelligence for literature reviews: opportunities and challenges (2024) · cites this paper as Justitia and Wang 2022
Citations
Cites
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,672
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865 · cited 1×
“More recently, BERTScore computes semantic similarities using contextual embeddings [39].”
MethodMeteor Universal: Language Specific Translation Evaluation for Any Target Language
Denkowski, M., & Lavie, A. (2014) · Proceedings of the Ninth Workshop on Statistical Machine Translation · doi:10.3115/v1/w14-3348 · Cited by 1,952 · cited 1×
“Meteor [35] and BLEU [36] are two well-known machine translation assessment measures that also work well in text summarization.”
BackgroundLexRank: Graph-based Lexical Centrality as Salience in Text Summarization
Erkan, G., & Radev, D. R. (2004) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1523 · Cited by 1,700 · cited 1×
“LextRank, introduced by [32], uses the random walk approach to rank sentences in a graph-based summarization.”
MethodAutomatic text summarization: A comprehensive survey
El-Kassas, W. S., Salama, C. R., Rafea, A. A., & Mohamed, H. K. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113679 · Cited by 809 · cited 3×
“In the extractive approach, the summarizer combines the most important sentences from the original article [4].”
BackgroundMethodTextRank: Bringing Order into Texts
Mihalcea, R., & Tarau, P. (2004) · University of North Texas Digital Library (University of North Texas) · Cited by 632
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance
Zhao, W., Peyrard, M., Liu, F., Gao, Y., Meyer, C. M., & Eger, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1053 · Cited by 490 · cited 1×
“MoverScore [38] also combines contextualized embeddings with Earth Mover's Distance, which calculates the semantic distance between texts.”
BackgroundWikipedia-based Semantic Interpretation for Natural Language Processing
Gabrilovich, E., & Markovitch, S. (2009) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2669 · Cited by 377 · cited 1×
“Unfortunately, the lexical character in the thesaurus is powerful where the individual words are used to find the high similarity values of other similar words [22].”
BackgroundMEAD - A Platform for Multidocument Multilingual Text Summarization
Radev, D., Allison, T., Blair-Goldensohn, S., Blitzer, J. C., Çelebi, A., Dimitrov, S., Drábek, E. F., Hakim, A., Lam, W. P., Liu, D., et al. (2004) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2nevc4eu3swt · Cited by 335
Yasunaga, M., Kasai, J., Zhang, R., Fabbri, A. R., Li, I., Friedman, D., & Radev, D. R. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017386 · Cited by 192 · cited 2×
“[23] - - X X - X X - - ScisummNet [40] corpus 1000 set papers 8 set target papers from computer science field INPUT SYSTEM Papers with their citation sentences Abstractive Approach”
BackgroundDeep reinforcement and transfer learning for abstractive text summarization: A review
Alomari, A., Idris, N. B., Sabri, A. Q. M., & Alsmadi, I. M. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101276 · Cited by 170
Automatic summarization of scientific articles: A survey
Altmami, N. I., & Menaï, M. E. B. (2020) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2020.04.020 · Cited by 150 · cited 3×
“Summarizing scientific articles automatically becomes urgent and essential to reduce time and energy [9].”
BackgroundMethodScientific document summarization via citation contextualization and scientific discourse
Cohan, A., & Goharian, N. (2017) · International Journal on Digital Libraries · doi:10.1007/s00799-017-0216-8 · Cited by 115 · cited 2×
“The severe impact is changing others' claims into facts even though assumptions in the previous study [15].”
BackgroundLiterature review writing: how information is selected and transformed
Jaidka, K., Khoo, C. S. G., & Na, J. (2013) · Aslib Journal of Information Management · doi:10.1108/00012531311330665 · Cited by 64 · cited 1×
“Jaidka et al. [19] reported that there were two types of related work written by the majority of researches: integrative and descriptive related work.”
BackgroundExplaining Relationships Between Scientific Documents
Luu, K., Wu, X., Koncel-Kedziorski, R., Lo, K., Cachola, I. A., & Smith, N. A. (2021) · doi:10.18653/v1/2021.acl-long.166 · Cited by 57 · cited 1×
“Luu et al. [43] It experimented in two ways, by fine-tuning and based on information retrieval.”
BackgroundSummarizing figures, tables, and algorithms in scientific publications to augment search results
Bhatia, S., & Mitra, P. (2012) · ACM Transactions on Information Systems · doi:10.1145/2094072.2094075 · Cited by 55 · cited 1×
“Several other studies take a different approach to producing summarization using article elements, such as figures and tables [14] and citation sentences [15]– [18].”
BackgroundTowards Automated Related Work Summarization
Hoang, C. D. V., & Kan, M. (2010) · National University of Singapore · Cited by 53
BACO: A Background Knowledge- and Content-Based Framework for Citing Sentence Generation
Ge, Y., Dinh, L., Liu, X., Su, J., Lu, Z., Wang, A., & Diesner, J. (2021) · doi:10.18653/v1/2021.acl-long.116 · Cited by 53 · cited 1×
“Ge et al. [42] It incorporated background knowledge, salience estimation, and citation function to create a citation network , determine what to cite , and categorize why to cite , respectively.”
BackgroundAutomatic generation of related work through summarizing citations
Chen, J., & Zhuge, H. (2017) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.4261 · Cited by 49 · cited 7×
“Chen and Zhuge [7] created RWS-Cit, a related work section generator, citing sentences from referenced paper.”
BackgroundMethodResultInfluentialAutomatic Generation of Related Work Sections in Scientific Papers: An Optimization Approach
Hu, Y., & Wan, X. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1170 · Cited by 48 · cited 3×
“Chen and Zhuge [7], [26] compared their proposed systems to the ReWoS system [5] and ARWG system [6].”
BackgroundMethodResultData-driven Summarization of Scientific ArticlesPreprint
Nikolov, N. I., Pfeiffer, M., & Hahnloser, R. H. R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.08875 · Cited by 46 · cited 1×
“The hybrid approach, on the other hand, combines two methods: extractive and abstractive [9] Early conventional studies in the scientific article domain focused on developing systems that can automatically generate title [11], abstract [12], introduction [8], and presentation slide [13].”
MethodSummarization of Scientific Paper Through Reinforcement Ranking on Semantic Link Network
Sun, X., & Zhuge, H. (2018) · IEEE Access · doi:10.1109/access.2018.2856530 · Cited by 46 · cited 1×
“In related work summarization, LSA is widely used to calculate semantic relations between sentences and store them in a semantic network database [25] or graph relation [7][26].”
MethodSummarization of scientific documents by detecting common facts in citations
Chen, J., & Zhuge, H. (2013) · Future Generation Computer Systems · doi:10.1016/j.future.2013.07.018 · Cited by 43 · cited 1×
“Chen and Zhuge [7], [21] utilized the thesaurus dictionary to expand the synonyms and hypernym of words.”
MethodNeural Related Work Summarization with a Joint Context-driven Attention Mechanism
Wang, Y., Liu, X., & Gao, Z. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1204 · Cited by 40 · cited 4×
“Wang et al. [27] recommend the neural seq2seq paradigm and attention mechanism to achieve a contextual relevance summary while maintaining coherence between the related work section and its target paper.”
BackgroundInfluentialAutomatic Generation of Citation Texts in Scholarly Papers: A Pilot Study
Xing, X., Fan, X., & Wan, X. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.550 · Cited by 39 · cited 1×
“Xing et al. [41] It applied implicit citation sentences to add information about the cited paper between explicit citation sentences.”
BackgroundAutomatic related work section generation: experiments in scientific document abstracting
AbuRa’ed, A., Saggion, H., Shvets, A., & Bravo, Á. (2020) · Scientometrics · doi:10.1007/s11192-020-03630-2 · Cited by 34 · cited 5×
“At the same time, AbuRa'ed et al. [28] expanded the document corpus by adding sources from the Open Academic Graph (OAG) and Microsoft Academic Graph (MAG) in addition to ACL Anthology as the main corpora.”
BackgroundMethodInfluentialChen, X., Alamro, H., Li, M., Gao, S., Zhang, X., Zhao, D., & Yan, R. (2021) · doi:10.18653/v1/2021.acl-long.473 · Cited by 34 · cited 1×
“The generated citation sentence may be from one scientific document, far from the real scenario Chen et al. [44] 1.”
BackgroundAutomatic paper writing based on a RNN and the TextRank algorithm
Wang, H., Hsiao, W.-C., & Chang, S.-H. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106767 · Cited by 32 · cited 3×
“The hybrid approach, on the other hand, combines two methods: extractive and abstractive [9] Early conventional studies in the scientific article domain focused on developing systems that can automatically generate title [11], abstract [12], introduction [8], and presentation slide [13].”
BackgroundMethodSelf-citationPutra, J. W. G., & Khodra, M. L. (2017) · Journal of ICT Research and Applications · doi:10.5614/itbj.ict.res.appl.2017.11.3.3 · Cited by 26 · cited 1×
“The hybrid approach, on the other hand, combines two methods: extractive and abstractive [9] Early conventional studies in the scientific article domain focused on developing systems that can automatically generate title [11], abstract [12], introduction [8], and presentation slide [13].”
MethodContextualizing Citations for Scientific Summarization using Word Embeddings and Domain Knowledge
Cohan, A., & Goharian, N. (2017) · Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3077136.3080740 · Cited by 23
Wang, P., Li, S., Zhou, H., Tang, J., & Wang, T. (2019) · IEEE Access · doi:10.1109/access.2019.2959056 · Cited by 21 · cited 5×
“ToC-RWG [2] with QueryTopicSum and the identified CTS (Cited Text Spans) as candidate sentences, an optimization framework that minimizes KL divergence is exerted to select the most representative sentences for related work generation.”
BackgroundMethodInfluentialCitation sentence identification and classification for related work summarization
Widyantoro, D. H., & Amin, I. (2014) · doi:10.1109/icacsis.2014.7065871 · Cited by 16 · cited 2×
“Altmami and Menai utilized datasets from research [29], selecting 20 article sets and their reference papers.”
BackgroundMethodLauscher, A., Glavaš, G., & Eckert, K. (2017) · MADOC (University of Mannheim) · Cited by 16
Abstractive Summarization Using Attentive Neural Techniques
Krantz, J., & Kalita, J. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08838 · Cited by 15 · cited 1×
“VERT score combines similarity and dissimilarity sub-score for measuring semantic similarity of summary pair and compared the generated summary with ground-truth summary [37].”
MethodAutomatically Generating Engaging Presentation Slide Decks.
Winters, T., & Mathewson, K. W. (2019) · Lirias (KU Leuven) · Cited by 8
CitationAS: A Summary Generation Tool Based on Clustering of Retrieved Citation Content.
Wang, J., Ma, S., & Zhang, C. (2017) · Cited by 6
Event-based summarization method for scientific literature
Zhang, J., Li, K., Yao, C., & Sun, Y. (2020) · Personal and Ubiquitous Computing · doi:10.1007/s00779-019-01301-5 · Cited by 6 · cited 1×
“A scientific paper reports study findings and a medium for researchers to communicate their ideas, methodology, findings, and conclusions on a specific research topic [1].”
BackgroundAutomatic Related Work Section Generation by Sentence Extraction and Reordering.
Deng, Z., Zeng, Z., Gu, W., Ji, J., & Bolin, H. (2021) · Cited by 5
Altmami, N. I., & Menaï, M. E. B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99344-7_23 · Cited by 3 · cited 2×
“Few researchers emphasize the importance of readability [23], [30], coherence [27], informative [23], contextual [27], relevance [27], [28], and redundancy [30].”
Background
Research works printed but not linked: 5
- lin, 2004, ROUGE: A Package for Automatic Evaluation of Summaries Chin-Yew, Text Summa-rization Branches out, 74
- team, 2021, Example of Methodology in a Research Paper (With Definition)
- 2022, Learn how to write a review of literature, The Writing Center University of Wisconsin - Madison
- steinberger, 2004, Using Latent Semantic Analysis in text summarization and summary evaluation, Proc ISIMVSP, 93
- chen, 2016, Summarizing Related Work through Citations, International Conference on Semantics Knowledge and Grid, 54
Cited by
Artificial intelligence for literature reviews: opportunities and challenges
Bolaños, F., Salatino, A., Osborne, F., & Motta, E. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10902-3 · Cited by 246
“This section will discuss some of the key research challenges identified from our analysis that the academic community will need to address in future work.”
InfluentialRashidov, A. (2024) · International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA) · doi:10.1109/hora61326.2024.10550536 · Cited by 3
“Also, research teams are faced with the need to make sound and effective decisions when analyzing multiple options, while at the same time considering their own interests and expertise in the field [4].”
Background
Metadata
- Authors
- Justitia, A., & Wang, H.-C.
- Affiliations
- Universitas Airlangga (ID)
- National Cheng Kung University (TW)
- Venue
- 2022 International Seminar on Intelligent Technology and Its Applications (ISITIA)
- Publisher
- IEEE
- Published
- 2022-07-20
- Pages
- 108-114
- OpenAlex
- W4292388001
- Cited by
- 2
- References
- 40
- Citations by year
- 2024: 2
- FWCI
- 0.28
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Seminar on Intelligent Technology and Its Applications (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- research paper writing, literature review automation, extractive summarization, abstractive summarization, research trends, evaluation metrics, research gaps
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.