The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index; 10 of the 21 sources its claims cite were checked for support.
TLDR: Extreme Summarization of Scientific Documents
Open accessAbstract
We introduce TLDR generation, a new form of extreme summarization, for scientific papers.TLDR generation involves high source compression and requires expert background knowledge and understanding of complex domain-specific language.To facilitate study on this task, we introduce SCITLDR, a new multi-target dataset of 5.4K TLDRs over 3.2K papers.SCITLDR contains both author-written and expert-derived TLDRs, where the latter are collected using a novel annotation protocol that produces high-quality summaries while minimizing annotation burden.We propose CATTS, a simple yet effective learning strategy for generating TLDRs that exploits titles as an auxiliary training signal.CATTS improves upon strong baselines under both automated metrics and human evaluations.Data and code are publicly available at https://github.com/allenai/scitldr. Dataset constructionOverview We introduce SCITLDR, a new multitarget dataset of 5,411 TLDRs over 3,229 scientific papers in the computer science domain.3 The training set contains 1,992 papers, each with a single gold TLDR.The dev and test sets contain 619 and 618 papers each, with 1,452 and 1,967 TLDRs, respectively.This is unlike the majority of existing 3 See Appendix Table 9 for full venue breakdown.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper introduces TLDR generation for scientific papers and releases SciTLDR, a multi-target dataset of 5,411 TLDRs over 3,229 computer science papers. It proposes CATTS, which uses paper titles as an auxiliary training signal, and reports improvements over strong extractive and abstractive baselines on automated and human evaluations.
Key findings
- SciTLDR contains 5,411 TLDRs over 3,229 papers: 1,992 training papers with one gold TLDR each, 619 dev papers with 1,452 TLDRs, and 618 test papers with 1,967 TLDRs. Its summaries average 21 words, with a compression ratio of 238.1 and 15.2% novel words.
- Author-written TLDRs average 18.9 words and peer-review-derived TLDRs average 22.9 words; lexical overlap is very low (1-, 2-, and 3-gram mean Jaccard indices of 15.0%, 2.5%, and 0.7%). TLDR-PR is more abstractive, with 20.2% novel words versus 9.6% for TLDR-Auth.
- CATTS improves over BART baselines: in abstract-only input, CATTS gains +0.5 Rouge-1 and CATTS_XSUM gains +1.8 Rouge-1; in AIC input, CATTS gains +2.0 Rouge-1 and CATTS_XSUM gains +0.9 Rouge-1. In AIC, CATTS reaches 44.9 Rouge-1, 22.6 Rouge-2, and 37.3 Rouge-L versus BART’s 42.9/20.8/35.1.
- Human evaluation found CATTS_XSUM more informative than BART_XSUM (MRR 0.54 vs 0.42), comparable to gold TLDR-Auth (0.53) but below TLDR-PR (0.60). Correctness did not differ: both models averaged 2.5, with 42 ties, 10 cases favoring BART_XSUM, and 12 favoring CATTS_XSUM.
Methods
- Dataset construction combined author-written TLDRs collected from OpenReview with expert-derived TLDR-PR summaries created by rewriting the first 128 words of peer review comments. Twenty-eight University of Washington undergraduate computer science students were recruited, 20 continued after initial training, and annotators were paid $20/hour; full texts were parsed with S2ORC.
- Baselines included extractive PACSUM, BERTSUMEXT, and MatchSum, plus abstractive BART-large and BART-large finetuned on XSUM. CATTS appends control codes for TLDR and title generation, trains with an additional 20K arXiv paper-title pairs, and upsamples TLDR instances; experiments use abstract-only and abstract/introduction/conclusion (AIC) input spaces.
- Evaluation uses max Rouge over multiple gold TLDRs per paper, plus human evaluation: nugget-based informativeness on 76 sampled gold papers and author-rated correctness from 29 unique authors covering 64 arXiv papers.
Limitations
- Stated in the paper: experiments are limited to abstract-only and AIC input spaces, though full texts are provided for future work; the idea of a TLDR may differ across academic disciplines and is left for future work; a small fraction (<5%) of papers lacked available PDFs and were included using abstract-only text.
- Evident from the text: the dataset is restricted to computer science papers, and the human correctness evaluation is small (29 authors, 64 papers) and found no difference between BART_XSUM and CATTS_XSUM.
- Evident from the text: informativeness analysis uses only 76 sampled gold papers, and automated evaluation relies on max Rouge over multiple gold TLDRs, which rewards matching any gold summary and may not fully capture factual correctness.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
35 Own result · 15 No basis given · 9 Own method · 9 Own interpretation · 5 Partly supported · 4 Sources not found · 2 Supported · 2 No full text of the sources · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“As evidenced by earlier work in summarization evaluation (Cohan and Goharian, 2016), variability in human-written summaries (Zechner, 1996; Harman and Over, 2004) can negatively impact the reliability of automated summarization metrics like Rouge (Lin, 2004).”
What the sources say
- Not matchedcites Cohan and Goharian, 2016
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Zechner, 1996 source
Section 4.3 reports mean interhuman precision and recall of 0.49 when each subject’s selected sentences were compared with the others, showing variability among human-produced extracts. The citation to Zechner is attached to that variability; the sentence cites other works for the evaluation and Rouge points. Page numbers are not available in the supplied copy. (LLM, conf=0.87)
- Not matchedcites Lin, 2004
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
Every source the paper cites for this claim was checked and supports it.
In the paper
“We demonstrate that control codes can be used to effectively incorporate scaffold tasks (e.g. title generation) for denoising autoencoders like BART (Lewis et al., 2020).”
What the sources say
- Supportedcites Lewis et al., 2020 source
Section 2 (Model; page not available) identifies BART as a denoising autoencoder, which is what the citation establishes in the citing sentence. The control-code result is presented as the citing authors’ own demonstration, not as a finding attributed to Lewis et al. (LLM, conf=0.92)
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is unlike the majority of existing summarization datasets that assume only one gold summary for a given document.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We can also observe through qualitative examples in Figure 3 how TLDR-Auth and TLDR-PR can differ greatly, even when they contain the same information content.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, TLDRPR focus more on articulating problems the paper addresses ( P ).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The training set contains 1,992 papers, each with a single gold TLDR.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Interestingly, TLDR-PR place less emphasis on R and V in favor of further methodological details in the paper D .”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we introduce SCITLDR, a new multi-target dataset of 5.4K TLDRs over 3.2K papers.”
- Also stated in: § 1 Introduction · p. 1 · plain
“we introduce SCITLDR, a new dataset of 5,411 TLDRs of computer science papers.”
- Also stated in: § 2 Dataset construction · p. 2 · plain
“We introduce SCITLDR, a new multitarget dataset of 5,411 TLDRs over 3,229 scientific papers in the computer science domain.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 Introduction · p. 1 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In fact, they are the most frequently co-occurring nuggets, appearing in 63% of TLDRAuth and 71% of TLDR-PR.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We introduce CATTS (Controlled Abstraction for TLDRs with Title Scaffolding), a simple yet effective method for learning to generate TLDRs.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Titles often contain key information about a paper.
- It may state two separate claims as one. 30
- Probably not a substantial claim. 28
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The goal of existing work in summarization of scientific documents is to generate abstracts or provide complimentary summaries to abstracts. (Collins et al., 2017; Cohan et al., 2018; Chandrasekaran et al., 2019; Yasunaga et al., 2019).”
What the sources say
- Not matchedcites Collins et al., 2017
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Cohan et al., 2018 source
The Introduction describes this paper’s model as generating summaries of scientific papers. The Data section says the papers’ abstracts serve as ground-truth summaries, supporting the claim’s abstract-generation alternative. (LLM, conf=0.89)
- Partly supportedcites Chandrasekaran et al., 2019 sourceExperimental — accuracy not yet measured
The Introduction (p. 1) describes citation-based mini-summaries, and the Task section (p. 2) asks systems to generate a structured summary of a scientific paper. It lists abstracts alongside community and human-written summaries, but does not say that the task aims to generate abstracts or that its summaries are intended to complement abstracts. It also does not characterize the goals of existing work generally. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Incorporating auxiliary scaffold tasks via multitask learning has been studied before for improving span-labeling and text classification (Swayamdipta et al., 2018b; Cohan et al., 2019).”
What the sources say
- Not matchedcites Swayamdipta et al., 2018b
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Cohan et al., 2019 source
The Introduction and §2.1 describe auxiliary structural scaffold tasks in a multitask framework for citation-intent classification. The Results (§4.3) report improved classification performance when the scaffolds are added. This supports the text-classification part of the sentence for which Cohan et al. is cited. (LLM, conf=0.91)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
Every source the paper cites for this claim was checked and supports it.
In the paper
“We use BART-large, which achieves state-of-the-art results in summarization on XSUM.”
What the sources say
- Supportedcites Lewis et al., 2020 source
Section 5.1 identifies the model used in the large-scale experiments as the large BART model. Section 5.3 reports that BART outperformed the best previous XSum summarization system by roughly 3.5 points on all ROUGE metrics, supporting the state-of-the-art claim. (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Similar to multitask learning, training on heterogenous data annotated with control codes has been shown to improve controlled generation in autoregressive language models (Keskar et al., 2019; ElSahar et al., 2020; Sudhakar et al., 2019; Li et al., 2020).”
What the sources say
- Supportedcites Keskar et al., 2019 source
The Introduction (p. 1) describes training CTRL on varied text sources with control codes derived from their associated structure. Language Modeling with CTRL (p. 2) identifies it as an autoregressive conditional language model, and Control Codes (p. 5) shows that the codes enable predictable variation and increasingly constrain generation. (LLM, conf=0.89)
- Partly supportedcites ElSahar et al., 2020 sourceExperimental — accuracy not yet measured
Section 4 (p. 4) says the authors enrich reviews with category-derived control codes during training. Section 7 (pp. 8–9) reports that removing the codes degrades summary quality and that generated summaries tend to follow appropriate codes. The paper supports improved control in its review-summarization model, but does not isolate heterogeneous training data as the reason for that improvement or establish the result for autoregressive language models generally. (LLM, conf=0.84)
- Partly supportedcites Sudhakar et al., 2019 sourceExperimental — accuracy not yet measured
Section 2.3.2 describes an autoregressive transformer whose B-GST input includes a special token for the target style, and Section 4.3 reports improved style-transfer results. But the paper does not show that training on heterogeneous data annotated with control codes caused that improvement; Section 2.3.3 describes reconstruction training for style transfer, not a comparison of heterogeneous control-code training against an alternative. (LLM, conf=0.86)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While our multi-target setting allows us to mitigate some of the limitations of Rouge (Conroy et al., 2011; Cohan and Goharian, 2016), we acknowledge that relying only on automated metrics is insufficient for evaluating the quality of the models.”
What the sources say
- Supportedcites Conroy et al., 2011 source
The citation is attached to the limitations of ROUGE, which the paper documents directly. In Section 3, the authors say the gap between automatic and manual evaluation persists and can force the use of manual evaluation; the Conclusion identifies linguistic quality as a deficiency of ROUGE. These passages also support the citing sentence’s broader caution about relying only on automated metrics for summary evaluation. (LLM, conf=0.91)
- Not matchedcites Cohan and Goharian, 2016
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also observe that SCITLDR is smaller in comparison to automatically collected datasets, such as XSUM and ArXiv, but is larger in comparison to other manually collected datasets, such as SciSummNet.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To address this, SCITLDR contains TLDRs written from the perspective of the author ('TLDRAuth') and TLDRs written from the perspective of the peer reviewer('TLDR-PR').”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“where the latter are collected using a novel annotation protocol that produces high-quality summaries while minimizing annotation burden.”
- Also stated in: § 1 Introduction · p. 2 · plain
“where the latter are collected using a novel annotation protocol that produces highquality summaries while avoiding the burden of reading the full paper.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
- Rests on an assumption: Scaling up data collection in a specialized scientific domain is costly and challenging.
- The authors state this as an assumption: The typical peer reviewer has carefully scrutinized the source paper and provided a faithful summary in their comment (often in the first paragraph).
- Also stated in: § 1 Introduction · p. 2 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Despite similarities in length, the 1-, 2-, and 3-gram mean Jaccard indices between TLDR-Auth and TLDR-PR are 15.0%, 2.5%, and 0.7%, respectively, indicating extremely little lexical overlap between the two sources of TLDRs.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The dev and test sets contain 619 and 618 papers each, with 1,452 and 1,967 TLDRs, respectively.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Most TLDRs contain between two to four nuggets (never all six), and will provide some indication of their subject area ( A ) and the paper's contributions ( C ).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe that SCITLDR is more abstractive compared with other scientific domain datasets but less abstractive compared with non-scientific domain datasets.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We observe that SCITLDR has short summaries, like XSUM and NewsRoom, with long source documents, like BigPatent and the other scientific-domain datasets. This results in a much higher compression ratio compared with existing datasets.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“TLDR-Auth tend to include results or scientific/theoretical findings ( R ) and often signal the value of their work ( V ) by describing their contributions as novel or their results as strong or state-of-the-art .”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, we note that TLDR-Auth are on average 18.9 words long, while TLDR-PR are slightly longer on average at 22.9 words.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“CATTS improves upon strong baselines under both automated metrics and human evaluations.”
- Also stated in: § 1 Introduction · p. 2 · boosted
“We show that CATTS applied to BART (Lewis et al., 2020), a state-of-the-art summarization model, results in performance improvement in both automated metrics and human evaluation.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § 1 Introduction · p. 2 · boosted
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“TLDR-PR is more abstractive with a novelty score of 20.2% compared with TLDRAuth with a novelty score of 9.6%, where novelty is computed as the percentage of words in the TLDR not in the source paper.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“While this up-sampling may indicate that CATTS is training on more TLDRs than BART, we allow BART training up to 20 epochs and it quickly overfits within a few epochs.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The average combined length of these contexts is 993 words and resulting compression ratio is 47.3, which is still higher than other datasets surveyed in Table 1.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe that MatchSum has the highest extractive performance, followed by BERTSUMEXT.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Curiously, despite both achieving the same Rouge-1, the generated TLDRs are quite different.”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We observe that increasing input space from abstract-only to AIC greatly improves PACSUM 15 performance but decreases performance of both BERTSUMEXT and MatchSum.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The 'two hidden layer' detail by BART XSUM is from the paper introduction and the 'defining the appropriate sampling distributions' from CATTS XSUM is from the conclusion.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For a given abstractive baseline BART or BART XSUM , our CATTS learning strategy results in improvements in both abstract-only and AIC settings.”
- Also stated in: § 6.1 Quantitative results · p. 8 · plain
“Comparing CATTS variants with their corresponding BART baselines, we observe that in the abstract-only setting, CATTS and CATTS XSUM achieve +0.5 and +1.8 Rouge-1, respectively. In the AIC setting, CATTS and CATTS XSUM achieve +2.0 and +0.9 Rouge-1, respectively.”
- Also stated in: § 8 Conclusion · p. 9 · plain
“We show that our approach improves over strong modeling baselines.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § 6.1 Quantitative results · p. 8 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our full text oracle achieves 54.5 Rouge-1, 30.6 Rouge-2, and 45.0 Rouge-L on the test set.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The average length of an abstract is 159 words and resulting compression ratio is 7.6.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“From Table 5, we see the systems all generate TLDRs of similar length to the average length reported in Table 1.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“PACSUM using the full text yields a Rouge-1 of 12.7, significantly worse than abstract-only.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“BART XSUM and CATTS XSUM both achieve Rouge-1 of 40.7 on this paper.”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Summarization in higher compression settings is challenging as it requires capturing more precisely the salient aspects of the document (Grusky et al.).”
What the sources say
- Not matchedcites Grusky et al.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“TLDR generation involves high source compression and requires expert background knowledge and understanding of complex domain-specific language.”
- Also stated in: § 1 Introduction · p. 1 · plain
“Writing a TLDR of a scientific paper requires expert background knowledge and understanding of complex domain-specific language to identify the salient aspects of the paper, while maintaining faithfulness to the source and correctness of the written summary.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
- Also stated in: § 1 Introduction · p. 1 · plain
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We manually assessed every written summary, discarding TLDRs that did not adhere to the guidelines, and allowed 20/28 students who performed well to continue work beyond the first hour.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Our approach addresses two main challenges: (1) the limited size of the training data and (2) the need for domain knowledge in order to write high-quality gold TLDRs.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is not unexpected because TLDR-PR are derived from peer review comments which themselves have already gone through one stage of abstraction.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, the multiple target summaries in SCITLDR reflect diverse perspectives and can be used to support summarization research into training and evaluation techniques previously unavailable with existing datasets.”
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“To address the issue of multiple hypothesis testing over Rouge scores, we perform a Holm-Bonferroni (Holm, 1979) 16 correction for determining significant p -values in Table 3.”
What the sources say
- Not matchedcites Holm, 1979
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“SCITLDR is a new form of extreme summarization focused on scientific papers.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Supervised models like BERTSUMEXT and BART are trained on SCITLDR and the best model checkpoint chosen using dev set loss.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, while our experiments are limited to abstract-only and AIC input spaces, we provide the full text of the source papers to support research into using longer input contexts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In this setting, we have both chosen the strongest baseline and controlled for XSUM pretraining.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Rests on an assumption: Human experts in computer science can reliably assess informativeness and correctness of system-generated TLDRs.
- Rests on an assumption: The 51 sampled papers used in human evaluation are representative of the SCITLDR test set.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We use the two-sided paired t-test against a null hypothesis of no difference to assess these differences.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since TLDRs often contain information spread across multiple sentences, we expect abstractive summarization methods to produce strong results for this task.”
boostedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, the multiple target summaries in SCITLDR reflect diverse perspectives and can be used to support summarization research into training and evaluation techniques previously unavailable with existing datasets.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Best hyperparameters for the models are selected based on dev set Rouge-1.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“BART XSUM focuses on the methodological contribution while CATTS XSUM focuses on a scientific finding.”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We received responses from 29 unique authors with annotations covering 64 arXiv papers.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Rests on an assumption: Original authors of papers can reliably assess the correctness of system-generated TLDRs for their papers.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Additionally, while our experiments are limited to abstract-only and AIC input spaces, we provide the full text of the source papers to support research into using longer input contexts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
Scaling up data collection in a specialized scientific domain is costly and challenging.
“Scaling up data collection in a specialized scientific domain is costly and challenging.”
The authors state this as an assumption.
The typical peer reviewer has carefully scrutinized the source paper and provided a faithful summary in their comment (often in the first paragraph).
“Assuming the typical peer reviewer has carefully scrutinized the source paper and provided a faithful summary in their comment (often in the first paragraph), domain experts can rewrite these comments into TLDRs.”
The paper relies on this without support.
Titles often contain key information about a paper.
“As titles often contain key information about a paper, we hypothesize that training a model to generate titles will allow it to learn how to locate salient information in the paper that will be also useful for generating TLDRs.”
The paper relies on this without support.
All papers have a title.
“In addition, all papers have a title, and thus we have an abundant supply of paper-title pairs for training.”
The paper relies on this without support.
Human experts in computer science can reliably assess informativeness and correctness of system-generated TLDRs.
“we also have human experts in computer science assess system-generated TLDRs under two criteria - informativeness and correctness.”
The paper relies on this without support.
Original authors of papers can reliably assess the correctness of system-generated TLDRs for their papers.
“we ask the original authors of papers to assess the correctness of our system-generated TLDRs.”
The paper relies on this without support.
The 51 sampled papers used in human evaluation are representative of the SCITLDR test set.
“We perform our human evaluation on BART XSUM and CATTS XSUM using the AIC input space on 51 sampled papers.”
The paper relies on this without support.
The 76 gold papers used for nugget analysis are representative.
“we perform the nuggetbased analysis for information content over systemgenerated TLDRs for the same 76 gold papers from Section 3.2.”
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 10×
“PACSUM is an extension of TextRank (Mihalcea and Tarau, 2004), utilizing directed edges and a pretrained BERT model as a sentence encoder.”
MethodInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526 · cited 1×
“To address the issue of multiple hypothesis testing over Rouge scores, we perform a Holm-Bonferroni (Holm, 1979)16 correction for determining significant p-values in Table 3.”
MethodRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 1×
“The first two use BERT-base and the last one uses RoBERTa-base (Liu et al., 2019).”
MethodExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
(2004) · Cited by 9,718
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · cited 5×
“We particularly focus on BART-large, a state-of-the-art pretrained model with strong results in text generation and summarization (Lewis et al., 2019).”
BackgroundMethodInfluentialTextRank: Bringing Order into Text
Mihalcea, R. F., & Tarau, P. (2004) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 3,631
Research works printed but not linked: 3
- Paul Over. 2003. An introduction to duc 2003: Intrinsic evaluation of generic news text summarization systems. In Proceedings of Document Understanding Conference 2003 .
- In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages 32693279, Hong Kong, China. Association for Computational Linguistics.
- Swabha Swayamdipta, Sam Thomson, Kenton Lee, Luke Zettlemoyer, Chris Dyer, and Noah A. Smith. 2018b. Syntactic scaffolds for semantic structures. In EMNLP .
Cited by
Sorted by citations
Text Data Augmentation for Deep Learning
Shorten, C., Khoshgoftaar, T. M., & Furht, B. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00492-0 · Cited by 1,852
“Cachola et al. [3] were able to collect a dataset of 5.4K Too Long; Didn’t Read (TLDR) summaries of 3.2K machine learning papers.”
Bulathwela, S., Pérez-Ortiz, M., Holloway, C., Cukurova, M., & Shawe-Taylor, J. (2024) · Sustainability · doi:10.3390/su16020781 · Cited by 269
“The generative capabilities of linguistic content (e.g., automatic question generation [41,42], personalised feedback [43] and summarisation [44]) have seen significant progress.”
SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization
Laban, P., Schnabel, T., Bennett, P. N., & Hearst, M. A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00453 · Cited by 238
“Recent efforts to expand the application of summarization to new domains such as legal (Kornilova and Eidelman, 2019) or scholarly (Cachola et al., 2020) text will hopefully lead to the study of inconsistency detection in these novel domains, and perhaps even out of summarization on tasks such as text simplification,…”
BackgroundCyberbullying detection for low-resource languages and dialects: Review of the state of the art
Mahmud, T., Ptaszynski, M. E., Eronen, J., & Masui, F. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103454 · Cited by 181
Augmenting Scientific Papers with Just-in-Time, Position-Sensitive Definitions of Terms and Symbols
Head, A., Lo, K., Kang, D., Fok, R., Skjonsberg, S., Weld, D. S., & Hearst, M. A. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445648 · Cited by 91
“angers [26], and crowds [39] to annotate and discuss passages of papers. Other approaches to saving the scientist time include tools to support literature search (e.g., [77, 80, 109]), summarize text [10, 87], or rewrite passages in simpler language [46]. 2.3.2 Links within Papers. Reading interfaces can also assist r…”
BackgroundInfluentialSelf-citationMSˆ2: Multi-Document Summarization of Medical Studies
DeYoung, J., Beltagy, I., van Zuylen, M., Kuehl, B., & Wang, L. L. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.594 · Cited by 120
“…Li et al., 2016), relation extraction (Jain et al., 2020; Luan et al., 2018; Kringelum et al., 2016), QA (Abacha et al., 2019), NLI (Romanov and Shivade, 2018; Khot et al., 2018), summarization (Cachola et al., 2020; Chandrasekaran et al., 2019), claim verification (Wadden et al., 2020), and more.”
BackgroundQuantifying the use and potential benefits of artificial intelligence in scientific research
Gao, J., & Wang, D. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02020-5 · Cited by 81
“AI may, for example, optimize the research process by powering new tools and systems that improve the efficiency of doing science, including improving access to information, reducing the knowledge burden, guiding human intuition, automating routine research tasks and more 115,116 .”
The Semantic Scholar Open Data Platform
Kinney, R., Anastasiades, C., Authur, R., Beltagy, I., Bragg, J., Buraczynski, A., Cachola, I., Candra, S., Chandrasekhar, Y., Cohan, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.10140 · Cited by 64
Self-citationAspect-Controllable Opinion Summarization
Amplayo, R. K., Angelidis, S., & Lapata, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.528 · Cited by 71
“Text summarization models, both extractive (Narayan et al., 2018; Zheng and Lapata, 2019; Cachola et al., 2020) and abstractive (See et al.”
Automated Lay Language Summarization of Biomedical Scientific Reviews
Guo, Y., Qiu, W., Wang, Y., & Cohen, T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16089 · Cited by 61
“…have been achieved through the development of high-quality datasets and evaluation tasks, including but not limited to abstract generation of academic papers (Cohan et al. 2018), citation sentence generation (Luu et al. 2020) or extreme summarization of the entire document (Cachola et al. 2020).”
BackgroundRetrieval augmentation of large language models for lay language generation
Guo, Y., Qiu, W., Gondy Leroy, A., Wang, S., & Cohen, T. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104580 · Cited by 62
“Text summarization is a widely-studied research topic Cohan et al. (2018); Cachola et al. (2020); Devaraj et al. (2022).”
BackgroundKang, H. B., Chang, J. C., Kim, Y., & Kittur, A. (2022) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3526113.3545660 · Cited by 48
Chang, J. C., Zhang, A. X., Bragg, J., Head, A., Lo, K., Downey, D., & Weld, D. S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580847 · Cited by 47
“The Paper Cards include the title, authors, publication year, abstract, abstract summary [9], and citation count of its corresponding inline citation.”
BackgroundSelf-citationMaking Science Simple: Corpora for the Lay Summarisation of Scientific Literature
Goldsack, T., Zhang, Z., Lin, C., & Scarton, C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.724 · Cited by 54
Automated scholarly paper review: Concepts, technologies, and challenges
Lin, J., Song, J., Zhou, Z., Chen, Y., & Shi, X. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101830 · Cited by 37
ReviewRobot: Explainable Paper Review Generation based on Knowledge Synthesis
Wang, Q., Zeng, Q., Huang, L., Knight, K., Ji, H., & Rajani, N. F. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.44 · Cited by 38
“, 2019b), paper generation (Wang et al., 2018b, 2019; Cachola et al., 2020) , and abstractive summarization (Gu et al.”
Relatedly: Scaffolding Literature Reviews with Existing Related Work Sections
Palani, S., Naik, A., Downey, D., Zhang, A. X., Bragg, J., & Chang, J. C. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580841 · Cited by 37
“She clicks some of the references to see their metadata, including title, abstract, TLDR [8], authors, publication year, conference, and citation count, and she collects some of the ones she wants to read later by clicking “Copy” .”
BackgroundMethodInfluentialSummEdits: Measuring LLM Ability at Factual Reasoning Through The Lens of Summarization
Laban, P., Kryściński, W., Agarwal, D., Fabbri, A. R., Xiong, C., Joty, S., & Wu, C.-S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.600 · Cited by 44
“With recent progress in generation capabilities of LLMs, automatic summarization is making its appearance in practical information consumption situations such as summarizing work meetings (Arabzadeh et al., 2022), health records (Jain et al., 2022), or scientific documents (Cachola et al., 2020).”
BackgroundMethodEUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domain
Aumiller, D., Chouhan, A. P. S., & Gertz, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.519 · Cited by 61
D2S: Document-to-Slide Generation Via Query-Based Text Summarization
Sun, E., Hou, Y., Wang, D., Zhang, Y., & Wang, N. X. R. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.111 · Cited by 48
“Although there has been increasing interest in constructing large-scale single-document text summarization corpora (CNN/DM (Hermann et al., 2015), Newsroom (Grusky et al., 2018), XSum (Narayan et al., 2018), TLDR (Cachola et al., 2020)) and developing various approaches to address this task (Pointer Generator (See et…”
BackgroundMethodBursting Scientific Filter Bubbles: Boosting Innovation via Novel Author Discovery
Portenoy, J., Radensky, M., West, J. D., Horvitz, E., Weld, D. S., & Hope, T. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501905 · Cited by 38
“For additional context, multiple participants expressed interest in having abstracts available, and P15 suggested including automated summaries [7].”
BackgroundSelf-citationInvestigating Entropy for Extractive Document Summarization
Khurana, A., & Bhatnagar, V. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115820 · Cited by 32
“…the earlier empirical observation that Abstract, Introduction, and Conclusion sections of the scientific articles are germane for summary (Kavila & Radhika, 2015; Cachola et al., 2020), we selectively target these sections for generic, concise, and informative sentences to be included in summaries.”
BackgroundThe Semantic Scholar Academic Graph (S2AG)
Wade, A. D. (2022) · The Web Conference (WWW) · doi:10.1145/3487553.3527147 · Cited by 28
Applying Transformer-Based Text Summarization for Keyphrase Generation
Glazkova, A. V., & Morozov, D. A. (2023) · Lobachevskii Journal of Mathematics · doi:10.1134/s1995080223010134 · Cited by 37
“For example, control codes were used for the joined generating titles and short summaries (TLDRs) for scientific papers in [7].”
MethodExplaining Relationships Between Scientific Documents
Luu, K., Wu, X., Koncel-Kedziorski, R., Lo, K., Cachola, I. A., & Smith, N. A. (2021) · doi:10.18653/v1/2021.acl-long.166 · Cited by 27
“Work in this area has mostly used used abstracts or peer reviews as targets (Cachola et al., 2020; Cohan et al., 2018; Jaidka et al., 2017).”
MethodSelf-citationTraditional and context-specific spam detection in low resource settings
Kawintiranon, K., Singh, L., & Budak, C. (2022) · Machine Learning · doi:10.1007/s10994-022-06176-x · Cited by 24
“BERT has been used successfully for other learning tasks, including sentiment analysis (Munikar et al., 2019), natural language inference (Hossain et al., 2020) and document summarization (Cachola et al., 2020).”
MethodEmploying Argumentation Knowledge Graphs for Neural Argument Generation
Al‐Khatib, K., Trautner, L., Wachsmuth, H., Hou, Y., & Stein, B. (2021) · doi:10.18653/v1/2021.acl-long.366 · Cited by 22
“…’mariguana»positive»problems’] @ Animal studies suggests ...’<|endoftext|>” While this way of matching and encoding has limitations, it has shown good results in practice when used with pre-trained neural models (Wit-teveen and Andrews, 2019; Cachola et al., 2020).”
ResultBreaking Down Walls of Text: How Can NLP Benefit Consumer Privacy?
Ravichander, A., Black, A. W., Norton, T. B., Wilson, S., & Sadeh, N. (2021) · doi:10.18653/v1/2021.acl-long.319 · Cited by 27
“…amounts of text in domains such as scientific publications and news (Jain et al., 2020; Cachola et al., 2020; Kang et al., 2018; Rush et al., 2015; See et al., 2017), with several practical tools based on these technologies helping users every day (Cachola et al., 2020; TLDR, 2021; News, 2021).”
BackgroundMethodInfluentialCausal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP
Jin, Z., von Kügelgen, J., Ni, J., Vaidhya, T., Kaushal, A., Sachan, M., & Schölkopf, B. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.748 · Cited by 21
“For example, summarization data collected from the TL;DR of scientific papers SciTldr (Cachola et al., 2020) should be causal, as the TL;DR summaries on OpenReview (some from authors when submitting the paper, others derived from the beginning of peer reviews) were likely composed after the original papers or reviews…”
MethodStrategizing AI in Business and Education
Przegalińska, A., & Jemielniak, D. (2023) · Cambridge University Press eBooks · doi:10.1017/9781009243520 · Cited by 22
Control Prefixes for Parameter-Efficient Text Generation
Clive, J., Cao, K., & Rei, M. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.31 · Cited by 50
“This strategy was inspired by previous work which incorporates auxiliary scaffold tasks in multitask learning to improve span-labeling and text classification (Swayamdipta et al., 2018; Cohan et al., 2019; Cachola et al., 2020).”
MethodBringing Structure into Summaries: a Faceted Summarization Dataset for Long Scientific Documents
Meng, R., Thaker, K. M., Zhang, L., Dong, Y., Yuan, X., Wang, T., & He, D. (2021) · doi:10.18653/v1/2021.acl-short.137 · Cited by 33
“, 2020), which has recently achieved SOTA performance on abstractive summarization tasks with scientific articles (Cachola et al., 2020).”
BackgroundMethodInfluentialControl Prefixes for Text Generation.Preprint
Clive, J., Cao, K., & Rei, M. (2021) · arXiv (Cornell University) · Cited by 19
Automatic Pull Request Title Generation
Zhang, T., Irsan, I. C., Thung, F., Han, D., Lo, D., & Jiang, L. (2022) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme55016.2022.00015 · Cited by 22
“Oracle Extraction scores have been commonly used in summarization literature for comparison purpose [18], [19].”
MethodRefocusing on Relevance: Personalization in NLG
Dudy, S., Bedrick, S., & Webber, B. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.421 · Cited by 17
“…of multiple reference outputs, as in such cases an implicit assumption is that the references should typically exhibit minor linguistic variation, rather than summaries that vary substantially in their con- tents (Cachola et al., 2020; See et al., 2017; Grusky et al., 2018; Harman and Over, 2004).”
BackgroundTLDW: Extreme Multimodal Summarization of News Videos
Tang, P., Hu, K., Zhang, L., Luo, J., & Wang, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3296196 · Cited by 25
“Cachola et al. (2020) introduced multitask learning and incorporated the title generation as a scaffold task to improve the learning ability regarding the salient information.”
BackgroundSurveying the landscape of text summarization with deep learning: A comprehensive review
Wang, G., & Wu, W. (2023) · Discrete Mathematics Algorithms and Applications · doi:10.1142/s1793830923300047 · Cited by 19
A Novel Wikipedia based Dataset for Monolingual and Cross-Lingual Summarization
Fatima, M., & Strube, M. (2021) · doi:10.18653/v1/2021.newsum-1.5 · Cited by 14
Automated Literature Review Using NLP Techniques and LLM-Based Retrieval-Augmented Generation
Ali, N. F., Mohtasim, M. M., Mosharrof, S., & Krishna, T. G. (2024) · International Conference on Innovations in Science, Engineering and Technology (ICISET) · doi:10.1109/iciset62123.2024.10939517 · Cited by 15
“The SciTLDR dataset [3] is selected for this research work.”
Entity-Driven Fact-Aware Abstractive Summarization of Biomedical Literature
Alambo, A., Banerjee, T., Thirunarayan, K., & Raymer, M. L. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956656 · Cited by 13
“Neural abstractive summarization is well explored for summarization of news articles [1]–[5], and scientific articles [6]– [8].”
BackgroundCovSumm: an unsupervised transformer-cum-graph-based hybrid document summarization model for CORD-19
Karotia, A., & Susan, S. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-023-05291-3 · Cited by 13
FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge
Feng, S., Balachandran, V., Bai, Y., & Tsvetkov, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.59 · Cited by 40
EntSUM: A Data Set for Entity-Centric Extractive Summarization
Maddela, M., Kulkarni, M., & Preoţiuc-Pietro, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.237 · Cited by 14
Alambo, A., Banerjee, T., Thirunarayan, K., & Cajita, M. (2022) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc48229.2022.9871798 · Cited by 14
Mao, Y., Zhong, M., & Han, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.750 · Cited by 14
“C I T E S (prompting), without any fine-tuning, is also on par with the state-of-the-art method C ATTS (Cachola et al., 2020), while slightly worse than C ATTS XSUM , which pre-trains on the XSum dataset (Narayan et al., 2018) first.”
BackgroundMethodInfluentialGenerating informative conclusions for argumentative textsPreprint
Syed, S., Al-Khatib, K., Alshomary, M., Wachsmuth, H., & Potthast, M. (2021) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 12
Proceedings of the 4th New Frontiers in Summarization Workshop
(2023) · doi:10.18653/v1/2023.newsum-1 · Cited by 12
Cross-lingual extreme summarization of scholarly documents
Takeshita, S., Green, T., Friedrich, N., Eckert, K., & Ponzetto, S. P. (2023) · International Journal on Digital Libraries · doi:10.1007/s00799-023-00373-2 · Cited by 13
“SCITLDR [10], the basis for our work on multilingual summarization, presents a dataset for research papers (see Sect.3 for more details).”
BackgroundMethodResultInfluentialThe ACL OCL Corpus: Advancing Open Science in Computational Linguistics
Rohatgi, S., Qin, Y., Aw, B., Unnithan, N., & Kan, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.640 · Cited by 17
Lo, K., Chang, J. C., Head, A., Bragg, J., Zhang, A. X., Trier, C., Anastasiades, C., August, T., Authur, R., Bragg, D., et al. (2024) · Communications of the ACM · doi:10.1145/3659096 · Cited by 55
“These resources together open-source software, 3,4 AI models [11, 20, 42, 86], and open datasets [50, 61] to support continued work in this area.”
BackgroundMethodSelf-citationDataLab: A Platform for Data Analysis and Intervention
Yang, X., Fu, J., Yuan, W., Viswanathan, V., Liu, Z., Liu, Y., Neubig, G., & Liu, P. (2022) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2022.acl-demo.18 · Cited by 10
We Can Explain Your Research in Layman's Terms: Towards Automating Science Journalism at Scale
Dangovski, R., Shen, M., Byrd, D., Jing, L., Tsvetkova, D., Nakov, P., & Marin Soljacic, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17507 · Cited by 10
“Finally, we should mention multitask learning (Raffel et al. 2020; Lewis et al. 2019; Cachola et al. 2020) for improving summarization.”
Attribute-Sentiment-Guided Summarization of User Opinions From Online Reviews
Han, Y., Nanda, G., & Moghaddam, M. (2022) · Journal of Mechanical Design · doi:10.1115/1.4055736 · Cited by 11
MS2: Multi-Document Summarization of Medical Studies
DeYoung, J., Beltagy, I., van Zuylen, M., Kuehl, B., & Wang, L. L. (2021) · arXiv · doi:10.48550/arxiv.2104.06486 · Cited by 10
“…Li et al., 2016), relation extraction (Jain et al., 2020; Luan et al., 2018; Kringelum et al., 2016), QA (Abacha et al., 2019), NLI (Romanov and Shivade, 2018; Khot et al., 2018), summarization (Cachola et al., 2020; Chandrasekaran et al., 2019), claim verification (Wadden et al., 2020), and more.”
BackgroundCitance-Contextualized Summarization of Scientific Papers
Syed, S., Hakimi, A., Al‐Khatib, K., & Potthast, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.573 · Cited by 8
AI Transparency in Academic Search Systems: An Initial Exploration
Liu, Y., Sullivan, P., & Sinnamon, L. S. (2024) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1167 · Cited by 8
“A few systems (1, 5) include references to publications that describe the scientific process used to develop and evaluate system components (Cachola et al., 2020; Nicholson et al., 2021); however, these are not easily findable and may be outdated or only partially reflective of the current systems…”
SMSMO: Learning to generate multimodal summary for scientific papers
Zhong, X., Tan, Z., Gao, S., et al. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112908 · Cited by 8
Using Pre-Trained Transformer for Better Lay Summarization
Kim, S. (2020) · Workshop on Scholarly Document Processing (SDP) · doi:10.18653/v1/2020.sdp-1.38 · Cited by 8
“…scientific documents is limited in a sense, which the produced summary is either too concise to provide important information (Vasilyev et al., 2019; Cachola et al., 2020) or aiming to directly extract the content from abstract or citation sentences (Yasunaga et al., 2019), which mostly resembles…”
BackgroundMethodThe Effect of Pretraining on Extractive Summarization for Scientific Documents
Gupta, Y., Ammanamanchi, P. S., Bordia, S., Manoharan, A., Mittal, D., Pasunuru, R., Shrivastava, M., Singh, M., Bansal, M., & Jyothi, P. (2021) · Workshop on Scholarly Document Processing (SDP) · doi:10.18653/v1/2021.sdp-1.9 · Cited by 8
“We present results for both settings and use the splits specified in (Cachola et al., 2020).”
MethodInfluentialLee, S., Yang, K., Park, C., Sedoc, J., & Lim, H. (2021) · IEEE Access · doi:10.1109/access.2021.3124556 · Cited by 9
“However, most of the existing investigation efforts on abstractive text summarization have been concentrated on single-speaker documents, such as news articles (Paulus et al., 2017; See et al., 2017) and scientific documents (Cachola et al., 2020; Er-era et al., 2019).”
BackgroundSPEC: Summary Preference Decomposition for Low-Resource Abstractive Summarization
Chen, Y.-S., Song, Y.-Z., & Shuai, H.-H. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3230539 · Cited by 8
Atri, Y. K., Goyal, V., & Chakraborty, T. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599830 · Cited by 10
Scientific document processing: challenges for modern learning methods
Kashyap, A. R., Yang, Y., & Kan, M. (2023) · International Journal on Digital Libraries · doi:10.1007/s00799-023-00352-7 · Cited by 13
“Although earlier works used multitask learning for mainly classification tasks [32,163], it has found a resurgence in text generation [16,203].”
BackgroundMethodInfluentialTLDR9+: A Large Scale Resource for Extreme Summarization of Social Media Posts
Sotudeh, S., Deilamsalehy, H., Dernoncourt, F., & Goharian, N. (2021) · Workshop on New Frontiers in Summarization (NewSum) · doi:10.18653/v1/2021.newsum-1.15 · Cited by 10
“Our proposed datasets are more suited for the task of extreme summarization (Narayan et al., 2018; Cachola et al., 2020), where the task is to create a short one-sentence summary.”
MethodHe, J., Jiang, W., Chen, G.-B., et al. (2022) · Journal of Computer Science and Technology · doi:10.1007/s11390-022-2125-6 · Cited by 7
“[36] proposed TLDR, an extreme summarization of scientific documents.”
BackgroundLimGen: Probing the LLMs for Generating Suggestive Limitations of Research Papers
Faizullah, A. R. B. M., Urlana, A., & Mishra, R. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-70344-7_7 · Cited by 6
Using Generative AI to Enhance Psychometric Scale Development in Market Research
Keane, D., & McNaughton, R. B. (2025) · International Journal of Market Research · doi:10.1177/14707853251384769 · Cited by 7
“Recent advances, including SciBERT, Longformer, and ResearchRabbit, enable large-scale literature scans, clustering of related de fi nitions, and preliminary synthesis (Beltagy et al., 2019; Cachola et al., 2020; Hoffmann et al., 2024).”
Natural Language Processing For Automatic text summarization [Datasets] - Survey
Al-Banna, A. A., & Al-Mashhadany, A. K. (2022) · Wasit Journal of Computer and Mathematics Science · doi:10.31185/wjcm.72 · Cited by 6
Siriwardhana, S., Gupta, C., Kaluarachchi, T., Dissanayake, V., Ellawela, S., & Nanayakkara, S. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2286090 · Cited by 6
“…autobio-graphical and unstructured Reddit-title dataset, i.e., autobio-graphical-only , which is motivated by the transfer learning literature (Cachola et al., 2020; Gururangan et al., 2020), and (2) fine-tune with a mix distribution of structured news data and unstructured Reddit data, i.e.,…”
InfluentialA System for Information Extraction from Scientific Texts in Russian
Bruches, E. P., Mezentseva, A. A., & Batura, T. V. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-12285-9_15 · Cited by 6
What Can Natural Language Processing Do for Peer Review?
Kuznetsov, I., Afzal, O. M., Dercksen, K., Dycke, N., Goldberg, A., Hope, T., Hovy, D., Kummerfeld, J. K., Lauscher, A., Leyton-Brown, K., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.06563 · Cited by 6
Simple Yet Effective Synthetic Dataset Construction for Unsupervised Opinion Summarization
Shen, M., Ma, J., Wang, S., Vyas, Y., Dixit, K., Ballesteros, M., & Benajiba, Y. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.142 · Cited by 5
SumBlogger: Abstractive Summarization of Large Collections of Scientific Articles
Zakkas, P., Verberne, S., & Zavrel, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56027-9_23 · Cited by 7
Multi-task fine-tuning for generating keyphrases in a scientific domain
Glazkova, A. V., & Morozov, D. A. (2023) · International Conference on Information Technology and Nanotechnology (ITNT) · doi:10.1109/itnt57377.2023.10139061 · Cited by 6
“Cachola et al. [28] proposed CATTS, a multi-task learning strategy for generating short summaries (TLDRs) using titles and abstracts of scholarly documents.”
BackgroundMethodContributionSum: Generating Disentangled Contributions for Scientific Papers
Liu, M.-H., Yen, A.-Z., Huang, H., & Chen, H. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615115 · Cited by 5
“We also exploit key sentences in abstract, introduction and conclusion sections as they contain useful information for summarization [3].”
MethodGeneral then Personal: Decoupling and Pre-training for Personalized Headline Generation
Song, Y.-Z., Chen, Y.-S., Wang, L., & Shuai, H.-H. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00621 · Cited by 8
FrameSum: Leveraging Framing Theory and Deep Learning for Enhanced News Text Summarization
Zhang, X., Wei, Q., Zheng, B., Liu, J., & Zhang, P. (2024) · Applied Sciences · doi:10.3390/app14177548 · Cited by 5
Takeshita, S., Green, T., Friedrich, N., Eckert, K., & Ponzetto, S. P. (2022) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3529372.3530938 · Cited by 10
“For this, we build upon the original work from [5] and focus on abstractive summarization, since this has been shown to outperform extractive summarization in a variety of settings.”
BackgroundMethodResultInfluentialAutomatic Text Summarization using Document Clustering Named Entity Recognition
Senthamizh Selvan., R., & Arutchelvan, K. (2022) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2022.0130962 · Cited by 5
“The introduction of the Transformer encoder-decoder models briefly sparked [8], [9] news [10] and logical articles [11] and significant improvements.”
BackgroundHowlings, E., & Solymosi, R. (2023) · International journal of missing persons/International journal of missing persons · doi:10.55917/2769-7045.1003 · Cited by 5
Gomes, L. I. E. (2022) · Revista Ibero-Americana de Ciência da Informação · doi:10.26512/rici.v15.n1.2022.41490 · Cited by 5
Towards Automatic Comparison of Data Privacy Documents: A Preliminary Experiment on GDPR-like LawsPreprint
Kawintiranon, K., & Liu, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.10117 · Cited by 9
“Some research studies [3, 4] have proposed a transformer-based and n-gram-based model to summarize long scientific documents but this is limited to use for only scientific papers and incapable of legal document summarization.”
BackgroundMeta-research on COVID-19: An overview of the early trendsPreprint
Colavizza, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02961 · Cited by 8
Abstractive Summarization Model for Summarizing Scientific Article
Ülker, M., & Özer, A. B. (2024) · IEEE Access · doi:10.1109/access.2024.3420163 · Cited by 8
“In this study [16], they constructed a novel corpus (SciTLDR) consisting of scientific papers related to the computer science domain.”
MethodTraceable Texts and Their Effects: A Study of Summary-Source Links in AI-Generated Summaries
Kambhamettu, H., Flores, J., & Head, A. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719830 · Cited by 4
“From medical text [10] to legal documents [24], scientific literature [8] to online documents [40], interest is mounting in using AI to generate summaries to help users quickly understand the essence of lengthy, complicated texts.”
Multilingual Text Summarization in Healthcare Using Pre-Trained Transformer-Based Language Models
Käser, J., Nagy, T., Stirnemann, P., & Hanne, T. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.061527 · Cited by 5
“The finetuning of the model was performed on the Scitldr [44] dataset.”
Pre-training Meets Clustering: A Hybrid Extractive Multi-document Summarization Model
Karotia, A., & Susan, S. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-27409-1_48 · Cited by 4
CiteBench: A Benchmark for Scientific Citation Text Generation
Funkquist, M., Kuznetsov, I., Hou, Y., & Gurevych, I. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.455 · Cited by 11
Curriculum-guided Abstractive Summarization for Mental Health Online Posts
Sotudeh, S., Goharian, N., Deilamsalehy, H., & Dernoncourt, F. (2022) · International Workshop on Health Text Mining and Information Analysis (LOUHI) · doi:10.18653/v1/2022.louhi-1.17 · Cited by 4
Beyond Good Intentions: Reporting the Research Landscape of NLP for Social Good
Adauto, F. G., Jin, Z., Schölkopf, B., Hope, T., Sachan, M., & Mihalcea, R. F. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.31 · Cited by 8
Question-Answer Extraction from Scientific Articles Using Knowledge Graphs and Large Language Models
Azarbonyad, H., Zhu, Z. L., Cheirmpos, G., Afzal, Z., Yadav, V., & Tsatsaronis, G. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730068 · Cited by 3
Vidyarthi, A., Singh, M. K., & Moirangthem, D. S. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.131160 · Cited by 5
Fok, R., Soldaini, L., Trier, C., Bransom, E., MacMillan, K., Cheng, E. Y.-Y., Kambhamettu, H., Bragg, J., Lo, K., Hearst, M. A., et al. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3665648 · Cited by 7
“Some methods have even been proposed for generating extreme summaries (e.g., single sentence “too long; didn’t read” statements) from full papers [11].”
Self-citationMultimodal Paper Summarization with Hierarchical Fusion
Tan, Z., Zhong, X., & Chiu, B. (2024) · International Conference on Engineering and Emerging Technologies (ICEET) · doi:10.1109/iceet65156.2024.10913899 · Cited by 3
“…datasets tailored to various tasks, such as citation sentence generation [13], related work generation [14], abstract generation [15], extreme summarization (e.g., one-line summary for an entire paper) [16], and layman text generation (e.g., simplified abstracts for non-expert audiences) [17].”
Generating keyphrases for readers: A controllable keyphrase generation framework
Jiang, Y., Meng, R., Huang, Y., Lu, W. P., & Liu, J. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24749 · Cited by 4
“…encoder – decoder framework, the CKPG uses the keyphrase function as a control code, whose effectiveness for the autoregressive language models has been shown in (Cachola et al., 2020; Elsahar et al., 2020; Keskar et al., 2019), and then automatically generates keyphrases of the specified category.”
MethodInfluentialMUTANT: A Multi-sentential Code-mixed Hinglish Dataset
Gupta, R., Srivastava, V. P., & Singh, M. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.56 · Cited by 3
Applications of Shaped-Charge Learning
Galitsky, B. A. (2023) · Entropy · doi:10.3390/e25111496 · Cited by 4
MOOCMaven: Bridging M-Learning and Open Data for Enhanced Unified MOOC Exploration
Tatang, A. F., & Algarni, A. (2023) · International Journal of Interactive Mobile Technologies (iJIM) · doi:10.3991/ijim.v17i20.42445 · Cited by 3
Metadata
- Authors
- Cachola, I., Lo, K., Cohan, A., & Weld, D.
- Affiliations
- Allen Institute for Artificial Intelligence (US)
- University of Washington (US)
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2020
- Published
- 2020
- Pages
- 4766-4777
- OpenAlex
- W3021892802
- Semantic Scholar
- 3502a542b2e98d9094e1880a30f652d4170b9534
- Cited by
- 169
- References
- 51
- Citations by year
- 2026: 6 · 2025: 18 · 2024: 26 · 2023: 48 · 2022: 35 · 2021: 34 · 2020: 2
- FWCI
- 11.35
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (1936940, 2040196)
- University of Washington
- Office of Naval Research (N00014- 18-1-2193, N00014-18, N00014)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Linguistics and Language › Text Readability and Simplification
- Keywords
- annotation protocol, computer science literature
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-21, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-21 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
