From citation intent to knowledge contribution: Classifying what cited papers actually contribute

It may state two separate claims as one.

Duan and Tan (2026) noted the conflation between citation intent and cited content type, but their work remains centered on the citing context and does not analyze the cited paper's knowledge contributions directly.

“Although Duan and Tan (2026) noted this conflation between citation intent and cited content type, their work remains centered on the citing context and does not analyze the cited paper's knowledge contributions directly.”

• cites Duan and Tan (2026) sourceSupported

It may state two separate claims as one.

Different types of knowledge contributions are not equally relevant in impact evaluation, and their importance depends on the specific analytical purpose.

“Different types of knowledge contributions are not equally relevant in impact evaluation, and their importance depends on the specific analytical purpose.”

• cites Bezerra et al., 2026 sourceSupported

It may state two separate claims as one.

Chen et al. (2025), in scientometrics, infer contributions at the paper level from aggregated citation contexts, whereas our unit is the individual citation.

“Chen et al. (2025), in scientometrics, infer contributions at the paper level from aggregated citation contexts, whereas our unit is the individual citation.”

• cites Chen et al. (2025) sourceSupported

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Pramanick et al. (2025) proposed an eight-category taxonomy to extract and track author-stated contributions across five decades of NLP research.

“Pramanick et al. (2025) proposed an eight-category taxonomy to extract and track author-stated contributions across five decades of NLP research.”

• cites Pramanick et al. (2025) sourceSupported

Probably not a substantial claim.

Tahamtan and Bornmann (2019) estimated that only 10-15% of citations perform substantive knowledge transfer functions.

“Tahamtan and Bornmann (2019) estimated that only 10-15% of citations perform substantive knowledge transfer functions.”

• cites Tahamtan and Bornmann (2019) sourcePartly supported

  Section 3.1.1.3 (p. 27) defines influential citations as those that meaningfully use or extend cited work and reports study-specific rates of 10.3% and about 14–15%; Section 4 (p. 45) summarizes those findings. The paper does not present 10–15% as Tahamtan and Bornmann’s own estimate for citations generally—it reports estimates from the studies it reviews. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Citations appear primarily in the Introduction (44.0%) and Related Work (34.3%) sections.

“As illustrated in Figure 2(b), citations appear primarily in the Introduction (44.0%) and Related Work (34.3%) sections.”

Probably not a substantial claim.

The final sample comprises 4,249 papers.

“The final sample comprises 4,249 papers.”

It may state two separate claims as one.

ELMo ranks 7th overall but 181st as a Resource Tool.

“ELMo ranks 7th but 181st”

It may state two separate claims as one.

ROUGE ranks 3rd overall but 75th as a Resource Tool.

“ROUGE ranks 3rd overall but 75th as a Resource Tool”

It may state two separate claims as one.

Even at its best across the two settings, the strongest model reaches only 0.740 in macro F1 and still trails the SciBERT baseline.

“Even at its best across the two settings, the strongest model reaches only 0.740 in macro F1 and still trails the SciBERT baseline.”

It may state two separate claims as one.

Background dominates the corpus with 488,618 instances (60.91%), while Method (141,671, 17.66%), Empirical Finding (118,325, 14.75%), and Resource Tool (53,587, 6.68%) together account for less than 40%.

“Background dominates the corpus with 488,618 instances (60.91%), while Method (141,671, 17.66%), Empirical Finding (118,325, 14.75%), and Resource Tool (53,587, 6.68%) together account for less than 40%.”

It may state two separate claims as one.

BERT enters the Top 5 on all three knowledge contribution dimensions and has a 68.2% Method share.

“BERT enters the Top 5 on all three dimensions, yet its 68.2% Method share”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

The citation context exhibits a slightly right-skewed length distribution, with a median of 414 characters and a mean of 434 characters.

“The citation context, defined as the citing sentence together with its immediately preceding and following sentences, exhibits a slightly right-skewed length distribution, with a median of 414 characters and a mean of 434 characters (Figure 2(c)).”

It may state two separate claims as one.

At Top 75%, core knowledge contribution citation count captures all 174 award-winning papers while raw citation count misses 11.

“At Top 75%, it captures all 174 award-winning papers while raw citation count still misses 11, indicating that non-core knowledge contribution citations dilute the discriminative power of traditional citation counts.”

• AssumptionRests on an assumption: Award-winning papers serve as a valid proxy for high-impact papers.

Probably not a substantial claim. It may state two separate claims as one.

174 award-winning papers were matched, covering four categories: Best Paper, Outstanding Paper, Test of Time, and Best Resource.

“After merging and deduplication, 174 awardwinning papers were matched, covering four categories: Best Paper, Outstanding Paper, Test of Time, and Best Resource.”

• AssumptionRests on an assumption: Award-winning papers serve as a valid proxy for high-impact papers.

It may state two separate claims as one.

Fusing the two paths lifts macro F1 to 0.774, the best result among all compared models.

“Fusing the two paths lifts macro F1 to 0.774, the best result among all compared models.”

It may state two separate claims as one.

XGBoost achieves an accuracy of 0.685 but a macro F1 of only 0.353.

“As shown in Table 4, XGBoost achieves an accuracy of 0.685 but a macro F1 of only 0.353.”

It may state two separate claims as one.

Across all papers, the Spearman correlation with the overall rank is 0.393 for Resource Tool, 0.628 for Method, and 0.654 for Empirical Finding.

“Across all papers, the Spearman correlation with the overall rank is only 0.393 for Resource Tool, against 0.628 for Method and 0.654 for Empirical Finding.”

It may state two separate claims as one.

We develop a Dual-Path Fusion model that achieves 85.5% classification accuracy, outperforming mainstream LLMs.

“We develop a Dual-Path Fusion model that achieves 85.5% classification accuracy, outperforming mainstream LLMs.”

It may state two separate claims as one.

QwQ-32B, Claude Sonnet 4.5, and GPT-5-mini surpass BERT and SPECTER in terms of macro F1.

“QwQ-32B, Claude Sonnet 4.5, and GPT-5-mini surpass BERT and SPECTER in terms of macro F1.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

The yearly distribution of citations exhibits exponential growth.

“The yearly distribution exhibits exponential growth, reflecting the rapid development of the field.”

It may state two separate claims as one.

KCT outperforms Intent in all eight comparisons, and Intent never significantly exceeds KCT under the paired bootstrap test.

“KCT outperforms Intent in all eight comparisons, and Intent never significantly exceeds KCT under the paired bootstrap test.”

It may state two separate claims as one.

Citation intent classification infers the citing author's subjective purpose, so the same paper may receive different intent labels across argumentative scenarios, making such judgments inherently uncertain.

“The former infers the citing author's subjective purpose, so the same paper may receive different intent labels across argumentative scenarios, making such judgments inherently uncertain.”

Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.

By shifting the unit of analysis from citation frequency to knowledge content, the KCT opens new possibilities for more fine-grained research evaluation, and provides methodological support for objectively characterizing knowledge transfer in citations.

“By shifting the unit of analysis from citation frequency to knowledge content, the KCT opens new possibilities for more fine-grained research evaluation, and provides methodological support for objectively characterizing knowledge transfer in citations.”

Probably not a substantial claim. It may state two separate claims as one.

Small (1978) conceptualized cited papers as concept symbols but did not distinguish the knowledge these symbols carry.

“Small (1978) conceptualized cited papers as concept symbols but did not distinguish the knowledge these symbols carry.”

• cites Small (1978) sourceNot checked — no full text of the source

Probably not a substantial claim.

The feature window and the prediction target are strictly separated in time.

“The feature window and the prediction target are strictly separated in time.”

Probably not a substantial claim. It may state two separate claims as one.

Chen et al. (2025) classified citation contexts to profile the contributions of Nature and Science papers and found a notable discrepancy between actual contributions as perceived by peers and those self-identified by authors.

“Chen et al. (2025) classified citation contexts to profile the contributions of Nature and Science papers and examined their relationships with the division of labor among authors, finding a notable discrepancy between actual contributions as perceived by peers and those self-identified by authors.”

• cites Chen et al. (2025) sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

It may state two separate claims as one.

The process of deriving categories from the four stages is invariant while its output adapts to each field.

“The process is therefore invariant while its output adapts to each field.”

Probably not a substantial claim. It may be too vague to check against evidence.

Each direction points to the same underlying shift, a change in the unit of evaluation.

“Each direction points to the same underlying shift, a change in the unit of evaluation.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

The evaluation in our experiments kept the test set at its natural, imbalanced distribution and did not apply balanced test sets or resampling.

“The evaluation in our experiments kept the test set at its natural, imbalanced distribution and did not apply balanced test sets or resampling”

It may state two separate claims as one.

Knowledge contribution types recover differences in citation impact as distinct dimensions of influence, which lets a core-oriented indicator identify high-impact papers beyond citation volume.

“The knowledge contribution types instead recover them as distinct dimensions of influence, which is what lets a core-oriented indicator identify high-impact papers beyond citation volume.”

Probably not a substantial claim.

A citation may in principle involve several types of knowledge contribution at once.

“A citation may in principle involve several types at once, for instance when a cited work supplies both a method and a dataset.”

It may state two separate claims as one.

A rising citation count is easily read as continuing influence, yet a paper's core contribution share can fall even as its citations keep accumulating.

“A rising citation count is easily read as continuing influence, yet a paper's core contribution share can fall even as its citations keep accumulating.”

It may state two separate claims as one.

The gap between accuracy and macro F1 for XGBoost reflects accuracy inflation under class imbalance and shows that the task needs deep semantic understanding beyond handcrafted features.

“This gap reflects accuracy inflation under class imbalance and shows that the task needs deep semantic understanding beyond handcrafted features.”

It may state two separate claims as one.

Resource Tool citations form only a small share of the citation network yet dominate a distinct influence dimension, so the builders of datasets and toolkits are systematically undervalued whenever influence is read off a single number.

“They form only a small share of the citation network yet dominate a distinct influence dimension, so the builders of datasets and toolkits, whose own citation counts may be modest, are systematically undervalued whenever influence is read off a single number.”

It may state two separate claims as one.

Knowledge contribution classification offers a better basis for identifying high-impact papers and provides more stable predictive signals for scholarly dissemination than citation intent classification.

“We apply the KCT to impact assessment and dissemination prediction, demonstrating that knowledge contribution classification offers a better basis for identifying high-impact papers and provides more stable predictive signals for scholarly dissemination than citation intent classification.”

Probably not a substantial claim. It may be too vague to check against evidence.

It is this shift, rather than any single new indicator, that allows research evaluation to engage with the substance of knowledge transfer.

“It is this shift, rather than any single new indicator, that allows research evaluation to engage with the substance of knowledge transfer.”

It may state two separate claims as one.

Pramanick et al. (2025), in NLP, read author statements in abstracts, so their categories describe what authors claim.

“Pramanick et al. (2025), in NLP, read author statements in abstracts, so their categories describe what authors claim.”

• cites Pramanick et al. (2025) sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

The analysis covers only the field of computational linguistics.

“The analysis covers only the field of computational linguistics”

Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.

Existing citation classification research primarily infers citing motivations from the citing author's perspective, leaving the knowledge contributions of cited works largely unexamined.

“Yet existing citation classification research primarily infers citing motivations from the citing author's perspective, leaving the knowledge contributions of cited works largely unexamined.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

In the second stage, disagreements were resolved through group discussion among the annotators, with unresolved cases adjudicated by an expert outside the annotation team.

“In the second stage, disagreements were resolved through group discussion among the annotators, with unresolved cases adjudicated by an expert outside the annotation team.”

It may state two separate claims as one.

A paper can rank very differently across knowledge contribution dimensions, and these gaps show where its influence concentrates.

“A paper can therefore rank very differently across dimensions, and these gaps show where its influence concentrates.”

It may state two separate claims as one.

Anderson and Lemken (2023) showed that citation context analysis can reveal which specific knowledge claims from a cited work are drawn upon by citing authors, often finding that a large proportion of citations are peripheral rather than substantive.

“Anderson and Lemken (2023) showed that citation context analysis can reveal which specific knowledge claims from a cited work are drawn upon by citing authors, often finding that a large proportion of citations are peripheral rather than substantive.”

• cites Anderson and Lemken (2023)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper may give no reason why it holds.

Our categories are constructed from the Scientific Research Logic Model, a general account of empirical inquiry that decomposes any study into problem identification, problem modeling, method design and result analysis.

“Our categories are constructed from the Scientific Research Logic Model, a general account of empirical inquiry that decomposes any study into problem identification, problem modeling, method design and result analysis.”

It may state two separate claims as one.

The early-stage distribution of knowledge contribution types already predicts later dissemination reliably, whereas intent labels carry weaker early signal.

“The early-stage distribution of knowledge contribution types thus already predicts later dissemination reliably, whereas intent labels, subject to shifting citing contexts, carry weaker early signal.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

The annotators received rigorous training in the taxonomy and annotation guidelines, and were calibrated on 100 pre-annotated samples.

“The annotators received rigorous training in the taxonomy and the annotation guidelines, and were calibrated on 100 pre-annotated samples to align their understanding.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Annotation was completed by four doctoral students in information science, each with over three years of experience in literature analysis.

“Annotation was completed by four doctoral students in information science, each with over three years of experience in literature analysis.”

It may state two separate claims as one.

The overall Fleiss' κ among the four annotators was 0.775, with a 95% confidence interval of [0.727, 0.822] and significantly above zero (p < 0.001).

“The overall Fleiss' κ among the four annotators was 0.775, with a 95% confidence interval of [0.727, 0.822] and significantly above zero (𝑝 < 0.001 ), indicating that this level of agreement is statistically robust.”

Probably not a substantial claim.

To rule out the influence of model selection on conclusions and ensure a robust and fair comparison between the two frameworks, each task employs both a linear and a nonlinear model.

“To rule out the influence of model selection on conclusions and ensure a robust and fair comparison between the two frameworks, each task employs both a linear and a nonlinear model.”

It may state two separate claims as one.

The class-level one-vs-rest Cohen's κ averaged across annotator pairs is 0.88 for Method, 0.71 for Resource Tool, 0.67 for Empirical Finding, and 0.79 for Background.

“At the class level, the onevs-rest Cohen's κ averaged across annotator pairs is 0.88 for Method, 0.71 for Resource Tool, 0.67 for Empirical Finding, and 0.79 for Background, with all four reaching at least moderate agreement and the lowest value for Empirical Finding, consistent with its finer semantic boundary.”

Probably not a substantial claim. It may state two separate claims as one.

The sample retains only cited papers published no later than 2014 with at least five citations in the first three years, ensuring completeness of the observation period and estimation precision of early-stage label distributions.

“The sample retains only cited papers published no later than 2014 with at least five citations in the first three years, ensuring completeness of the observation period and estimation precision of early-stage label distributions.”

Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.

A more theory-oriented discipline may yield a Theoretical Contribution type.

“A more theoryoriented discipline, for instance, may yield a Theoretical Contribution type.”

Probably not a substantial claim. It may state two separate claims as one.

KCT extends the concept of cited papers as concept symbols into four semantically explicit dimensions, so citation networks become knowledge transmission networks in which the pathway of each knowledge type is independently analyzable.

“KCT extends that concept into four semantically explicit dimensions, so citation networks become knowledge transmission networks in which the pathway of each knowledge type is independently analyzable.”

Probably not a substantial claim. It may state two separate claims as one.

We employ nested 10-fold cross-validation in all experiments, where the outer 10 folds evaluate model performance and the inner 5 folds tune hyperparameters, with strict separation between the two layers to prevent information leakage.

“We employ nested 10-fold cross-validation in all experiments, where the outer 10 folds evaluate model performance and the inner 5 folds tune hyperparameters, with strict separation between the two layers to prevent information leakage.”

Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.

Existing studies remain limited to contribution identification and distribution description, and do not examine how different knowledge types differ in dissemination behavior and what this implies for research evaluation.

“However, these studies remain limited to contribution identification and distribution description, and do not further examine how different knowledge types differ in dissemination behavior and what this implies for research evaluation.”

Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.

The knowledge contribution perspective turns each citation into a traceable knowledge transmission behavior, so we know not only that a citation occurs, but also what type of knowledge is flowing.

“The deeper value of the knowledge contribution perspective is that it turns each citation into a traceable knowledge transmission behavior, so we know not only that a citation occurs, but also what type of knowledge is flowing.”

Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.

The KCT's classificatory logic is grounded in the general structure of the research process and thus holds potential for transfer to other domains.

“Although the KCT was constructed and validated using data drawn entirely from the ACL Anthology, its classificatory logic is grounded in the general structure of the research process and thus holds potential for transfer to other domains.”

Probably not a substantial claim. It may state two separate claims as one.

To ensure a fair comparison across papers published in different years, all observation windows are fixed at 10 years post-publication, and any citations received beyond this period are excluded.

“To ensure a fair comparison across papers published in different years, all observation windows are fixed at 10 years post-publication, and any citations received beyond this period are excluded.”

It may state two separate claims as one.

SciBERT attains the highest macro F1 because it is pre-trained from scratch on a scientific corpus with an in-domain vocabulary, and its token-level masked-language-modeling objective fits sentence-level discrimination better than document-embedding objectives.

“The SciBERT baseline attains the highest macro F1 with balanced Precision and Recall, because it is pre-trained from scratch on a scientific corpus with an in-domain vocabulary, and its token-level masked-language-modeling objective fits sentence-level discrimination better than document-embedding objectives.”

Probably not a substantial claim. It may be too vague to check against evidence.

Combining knowledge contribution and citation intent classifications would show both what knowledge a paper supplies and how that knowledge is used.

“Combining them together would show both what knowledge a paper supplies and how that knowledge is used.”

It may state two separate claims as one.

A paper can keep accumulating citations after its substantive influence has faded, and KCT detects this shift earlier.

“A paper can thus keep accumulating citations after its substantive influence has faded, and KCT detects this shift earlier.”

Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.

The ACL Anthology is the most comprehensive open-access repository of computational linguistics, offering full-text PDF access and standardized bibliographic metadata.

“We selected the ACL Anthology 1 as data source because it is the most comprehensive open-access repository of computational linguistics, offering full-text PDF access and standardized bibliographic metadata.”

It may state two separate claims as one.

Citation counts keep rising even as a paper's substantive role decays, so volume alone overstates lasting influence.

“Citation counts keep rising even as a paper's substantive role decays, so volume alone overstates lasting influence.”

Probably not a substantial claim.

In traditional impact assessment, every citation is treated as an equal unit, ignoring knowledge contribution differences among citations.

“In traditional impact assessment, every citation is treated as an equal unit, ignoring knowledge contribution differences among citations.”

Probably not a substantial claim.

Statistical significance of differences between frameworks is determined via paired bootstrap tests with 1,000 resamples.

“Statistical significance of differences between frameworks is determined via paired bootstrap tests with 1,000 resamples.”

It may state two separate claims as one.

Knowledge contributions and citation intents are two facets that Small (1982) had already placed within a single program of citation context analysis, yet later context-based evaluation has tended to treat them separately.

“Knowledge contributions and citation intents are two facets that Small (1982) had already placed within a single program of citation context analysis, yet later context-based evaluation has tended to treat them separately.”

• cites Small (1982)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

In the first stage, 300 instances were jointly annotated by all four annotators to assess agreement, while the remaining 1,700 instances underwent cross-annotation, with each instance independently completed by at least two annotators.

“In the first stage, 300 instances were jointly annotated by all four annotators to assess agreement, while the remaining 1,700 instances underwent cross-annotation, with each instance independently completed by at least two annotators.”