Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 24 of the 27 references it cites were found in a registry or an index, and 1 are not checked yet; none of the 16 sources its claims cite is checked for support yet.

Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora

Open access
Boudourides, Moses · 2026 · SSRN Electronic Journal

Abstract

What survives after temporal leakage is rigorously removed from citation link prediction? This paper answers that question across ten large-scale bibliographic corpora. We introduce a strongly leakage-mitigated methodological framework in which all network-derived quantities are computed exclusively from information available prior to the publication year of the citing paper. We formalize four forms of temporal leakage-future citation leakage, future topology leakage, metadata leakage, and semantic leakage-and apply this framework to ten bibliographic datasets retrieved from Dimensions.ai, spanning five broad scholarly domains: Science (Protein Folding, CRISPR), Engineering (Additive Manufacturing, Corrosion Protection), BioMed (Neuroblastoma, Osteosarcoma), Social Science (Income Inequality, Organizational Behavior), and Humanities (Film Studies, Memory Studies). Each corpus covers the period 1975-2024. The goal of this study is not to maximize citation prediction accuracy, but to decompose citation formation into distinct explanatory components under temporally rigorous conditions. Gradient Boosting achieves ROC-AUC scores ranging from 0.530 to 0.734 under temporally rigorous evaluation; these scores reflect the intrinsic difficulty of the prediction task under strict leakage-mitigated conditions, not a failure of the predictive model. We decompose citation formation into three components: semantic alignment (S), chronological distance (T), and citation-network prominence factors (N prom). Under this decomposition, chronological recency (T) dominates prediction across almost all corpora. Once the candidate set is restricted to plausible contemporaneous papers from the same topical corpus, directional semantic alignment adds little discriminative power beyond chronological recency: in four of the ten corpora, cited papers exhibit marginally lower directional semantic alignment to the citing paper than non-cited candidates. When citations occur despite weak semantic alignment, they tend to be temporally proximate, though they do not exhibit a consistent prominence pattern. We conclude that under leakage-controlled conditions, citation formation exhibits substantial domain-specific heterogeneity, but chronological distance factors consistently contribute more predictive signal than directional semantic alignment or citation-network prominence factors.

Checked 2026-11-18 · 1 reference not checked yet

Overview

CoChat Summary

TL;DR

This paper introduces a leakage-mitigated framework for citation link prediction and applies it to ten Dimensions corpora (1975–2024), decomposing citation formation into semantic alignment (S), chronological distance (T), and citation-network prominence (Nprom). Under temporally rigorous evaluation, chronological recency dominates prediction across almost all corpora, while directional semantic alignment adds little beyond T and Nprom contributes negligibly once T is known.

Key findings

  • Gradient Boosting achieves ROC-AUC from 0.530 (Corrosion Protection) to 0.734 (CRISPR) under temporally rigorous evaluation; temporal hold-out AUC is generally consistent with CV AUC, e.g. CRISPR 0.798, Protein Folding 0.745, Corrosion Protection 0.615.
  • In the S-T-Nprom decomposition, T alone is the strongest single predictor in all ten corpora, with AUC(T) ranging from 0.577 (Memory Studies) to 0.824 (Protein Folding); S alone is weak, AUC(S) 0.490–0.536; Nprom alone is near-chance in most corpora. ΔT > 0 in all ten corpora, ΔS < 0 in all ten, and ΔN|ST is near zero or negative in eight of ten, with only CRISPR (+0.004) and Memory Studies (+0.005) showing marginal positive Nprom contribution beyond S+T.
  • In four of ten corpora, cited papers exhibit marginally lower directional semantic alignment to the citing paper than non-cited candidates (mean difference < 0.01 on a 0–1 scale). When citations occur despite weak semantic alignment, they tend to be temporally proximate (ΔT < 0 in nine of ten corpora), with no consistent prominence pattern: seven of ten corpora show low-S citations as less prominent, while Income Inequality (+0.252) and Memory Studies (+0.094) show the opposite.
  • Top-k ranking on CRISPR degrades as the candidate pool widens: NDCG@10 is approximately 0.79 at k = 10 and 0.30 at k = 100; MRR is 0.7340 at k = 10, 0.5102 at k = 50, and 0.2241 at k = 100.

Methods

  • Design: a leakage-mitigated citation link prediction framework formalizing four temporal leakage types (future citation, future topology, metadata, semantic leakage); all network-derived features are computed from the temporally capped graph G_{t_A−1}, and negatives come from a constrained candidate space C(A) of non-cited papers within ±3 years of the true cited paper and in the same topical corpus, yielding balanced 1:1 positive-negative pairs.
  • Data: ten Dimensions.ai corpora spanning Science (Protein Folding, CRISPR), Engineering (Additive Manufacturing, Corrosion Protection), BioMed (Neuroblastoma, Osteosarcoma), Social Science (Income Inequality, Organizational Behavior), and Humanities (Film Studies, Memory Studies); journal articles 1975–2024; record counts range from 26,486 (Corrosion Protection) to 393,342 (Film Studies), and valid pair counts range from 112 (Protein Folding) to 754 (Film Studies).
  • Analysis: trained Logistic Regression, Linear SVM, Random Forest, and Gradient Boosting; used 5-fold stratified cross-validation, temporal hold-out (train ≤ 2015, test 2016–2020), rolling temporal windows, seven S/T/Nprom sub-models, SHAP and leave-one-out ablation, and top-k ranking with k ∈ {10, 50, 100} using MRR, NDCG@k, and Precision@k, with 95% bootstrap confidence intervals from 1000 resamples.

Limitations

  • Paper states: SBERT pre-training introduces systemic semantic leakage that cannot be perfectly mitigated without chronologically re-trained language models; top-k experiments approximate retrieval realism over constrained candidate pools rather than full-corpus recommendation; pair counts are small for sparse domains (e.g., Corrosion Protection, Income Inequality), with cross-validation standard deviations of ±0.018 and ±0.014; the study relies on Gradient Boosting over engineered features rather than deep learning; and the ten datasets are single keyword-based queries, not complete disciplinary corpora, so conclusions are conditional on the keyword-defined topical corpus and ±3-year hard-negative candidate space.
  • Paper states: co-citation overlap is typically zero because the citing paper A is newly published at t_A unless it previously existed as a preprint, so co-citation is retained as a structural baseline but contributes negligible predictive signal; D1 group sizes for semantically surprising citations are small in some corpora (n = 6–11), so corpus-specific deviations should not be over-interpreted.
  • Evident from the text: the candidate-generation protocol restricts both topical and chronological distance variation, leaving relatively little semantic variation to discriminate cited from non-cited papers; the paper offers this as a plausible explanation for the negative ΔS result and notes that the S-T-Nprom profiles should not be interpreted as direct causal models or generalized beyond the constrained evaluation setting.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2139/ssrn.7372978

Cites

24 of 27 references verified0 citation errors · 2 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Cross-validation ROC curves for all four classifiers across the ten disciplinary datasets. Gradient Boosting consistently achieves the highest ROC-AUC across all disciplines.
Figure 1 Cross-validation ROC curves for all four classifiers across the ten disciplinary datasets. Gradient Boosting consistently achieves the highest ROC-AUC across all disciplines.CC-BY · From the PDF
Figure 2 Citation link prediction AUC (Gradient Boosting, 5-fold CV) across all 10 Dimensions datasets, colour-coded by discipline.
Figure 2 Citation link prediction AUC (Gradient Boosting, 5-fold CV) across all 10 Dimensions datasets, colour-coded by discipline.CC-BY · From the PDF
Figure 3 Rolling chronological distance-window ROC-AUC for Gradient Boosting across the ten disciplinary datasets. Performance remains stable across all chronological distance windows for which sufficient data is available.
Figure 3 Rolling chronological distance-window ROC-AUC for Gradient Boosting across the ten disciplinary datasets. Performance remains stable across all chronological distance windows for which sufficient data is available.CC-BY · From the PDF
Figure 4 Temporal Generalization: CV AUC vs. Temporal Hold-out AUC (Test 2016-2020) across all 10 datasets.
Figure 4 Temporal Generalization: CV AUC vs. Temporal Hold-out AUC (Test 2016-2020) across all 10 datasets.CC-BY · From the PDF
Figure 5 Feature ablation across all 10 corpora: AUC drop when the specified feature is removed. Temporal gap (T) dominates in Science; semantic alignment (S) is largely redundant with T in Engineering and Social Science; a mixed T-S profile is observed in BioMed.
Figure 5 Feature ablation across all 10 corpora: AUC drop when the specified feature is removed. Temporal gap (T) dominates in Science; semantic alignment (S) is largely redundant with T in Engineering and Social Science; a mixed T-S profile is observed in BioMed.CC-BY · From the PDF
Figure 6 Normalized SHAP feature importance across all 10 datasets. Semantic alignment ('directional_similarity') and chronological distance ('citation_time_gap') together account for the overwhelming majority of feature importance across all domains.
Figure 6 Normalized SHAP feature importance across all 10 datasets. Semantic alignment ('directional_similarity') and chronological distance ('citation_time_gap') together account for the overwhelming majority of feature importance across all domains.CC-BY · From the PDF
Figure 7 Individual feature ablation profiles for all ten datasets, showing the absolute AUC drop when each feature is removed.
Figure 7 Individual feature ablation profiles for all ten datasets, showing the absolute AUC drop when each feature is removed.CC-BY · From the PDF
Figure 8 Individual normalized SHAP feature importance profiles for all ten datasets.
Figure 8 Individual normalized SHAP feature importance profiles for all ten datasets.CC-BY · From the PDF
Figure 9 S-T-N prom decomposition across all ten corpora. Panel A shows the AUC of all seven sub-models. Panel B shows the three marginal contribution deltas: ∆ S = AUC( S + T ) -AUC( T ), ∆ T = AUC( S + T ) -AUC( S ), and ∆ N | ST = AUC( S + T + N prom ) -AUC( S + T ). The green shaded region in Panel B highlights that ∆ S < 0 across all ten corpora.
Figure 9 S-T-N prom decomposition across all ten corpora. Panel A shows the AUC of all seven sub-models. Panel B shows the three marginal contribution deltas: ∆ S = AUC( S + T ) -AUC( T ), ∆ T = AUC( S + T ) -AUC( S ), and ∆ N | ST = AUC( S + T + N prom ) -AUC( S + T ). The green shaded region in Panel B highlights that ∆ S < 0 across all ten corpora.CC-BY · From the PDF
Figure 10 Semantically surprising citations across all ten corpora. Panel A: Cohen's d for S, T, and N prom (cited vs. non-cited pairs). Panel B: Mean chronological distance deviation (∆ T ) for the D1 group (lowest decile of directional similarity) relative to all positives. Panel C: Mean prominence deviation (∆ N prom ) for the D1 group. Negative ∆ T indicates that low-S citations are more temporally recent than average.
Figure 10 Semantically surprising citations across all ten corpora. Panel A: Cohen's d for S, T, and N prom (cited vs. non-cited pairs). Panel B: Mean chronological distance deviation (∆ T ) for the D1 group (lowest decile of directional similarity) relative to all positives. Panel C: Mean prominence deviation (∆ N prom ) for the D1 group. Negative ∆ T indicates that low-S citations are more temporally recent than average.CC-BY · From the PDF
Figure 11 Estimated logistic effects of directional semantic alignment (S) and citation time gap (T) on citation probability across three illustrative corpora. Curves are from separate univariate logistic regressions; intended for qualitative interpretation only. A positive slope for T indicates that papers with a larger citation time gap (i.e., older cited papers) have higher citation probability, consistent with the cumulative advantage of established literature.
Figure 11 Estimated logistic effects of directional semantic alignment (S) and citation time gap (T) on citation probability across three illustrative corpora. Curves are from separate univariate logistic regressions; intended for qualitative interpretation only. A positive slope for T indicates that papers with a larger citation time gap (i.e., older cited papers) have higher citation probability, consistent with the cumulative advantage of established literature.CC-BY · From the PDF

Verification

Claims the paper makes

0 Valid · 2 Not valid · 15 Not checked · 170 Own results and interpretations

72 more claims

Create a free account to see all 187 claims and to verify the 12 references still open.

See all claims — free account
  • It may state two separate claims as one. 71
  • Probably not a substantial claim. 49
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

5 of 6 · the rest aren’t loaded

1 more finding

Create a free account to see every finding of this check.

See all findings — free account

Citations

Cites

Show all 22 references

Research works printed but not linked: 4

  1. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. Introduction to Informa- tion Retrieval. Cambridge University Press, 2008.
  2. Robert K. Merton. The Matthew Effect in Science. Science, 159(3810):56-63, 1968. doi: 10.1126/science.159.3810. · doi:10.1126/science.159.3810
  3. David M. W. Powers. Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness and Correlation. Journal of Machine Learning Technologies, 2(1):37-63, 2011.
  4. Ellen M. Voorhees and Donna K. Harman. TREC: Experiment and Evaluation in Information Retrieval. MIT Press, 2005.

Cited by

Still being checked.

Metadata

Authors
Boudourides, M.
Affiliations
Northwestern University (US)
Corresponding author
Boudourides, Moses
Venue
SSRN Electronic Journal
Publisher
Elsevier BV
Published
2026
ISSN
1556-5068 (ISSN-L)
OpenAlex
W7169606375
Cited by
0
References
22
Type
posted content
Language
en
Open access
Locations
  • SSRN Electronic Journal (submittedVersion, cc-by)
  • Research Square (submittedVersion, cc-by)
Topics
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
  • Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
Keywords
temporal leakage, semantic alignment, citation networks, gradient boosting

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora. https://app.cochat.ai/papers/10.2139/ssrn.7372978
The paper
Boudourides, M. (2026). Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.7372978

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.