The DOI matches its Crossref record; 24 of the 27 references it cites were found in a registry or an index, and 1 are not checked yet; none of the 16 sources its claims cite is checked for support yet.
Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora
Open accessAbstract
What survives after temporal leakage is rigorously removed from citation link prediction? This paper answers that question across ten large-scale bibliographic corpora. We introduce a strongly leakage-mitigated methodological framework in which all network-derived quantities are computed exclusively from information available prior to the publication year of the citing paper. We formalize four forms of temporal leakage-future citation leakage, future topology leakage, metadata leakage, and semantic leakage-and apply this framework to ten bibliographic datasets retrieved from Dimensions.ai, spanning five broad scholarly domains: Science (Protein Folding, CRISPR), Engineering (Additive Manufacturing, Corrosion Protection), BioMed (Neuroblastoma, Osteosarcoma), Social Science (Income Inequality, Organizational Behavior), and Humanities (Film Studies, Memory Studies). Each corpus covers the period 1975-2024. The goal of this study is not to maximize citation prediction accuracy, but to decompose citation formation into distinct explanatory components under temporally rigorous conditions. Gradient Boosting achieves ROC-AUC scores ranging from 0.530 to 0.734 under temporally rigorous evaluation; these scores reflect the intrinsic difficulty of the prediction task under strict leakage-mitigated conditions, not a failure of the predictive model. We decompose citation formation into three components: semantic alignment (S), chronological distance (T), and citation-network prominence factors (N prom). Under this decomposition, chronological recency (T) dominates prediction across almost all corpora. Once the candidate set is restricted to plausible contemporaneous papers from the same topical corpus, directional semantic alignment adds little discriminative power beyond chronological recency: in four of the ten corpora, cited papers exhibit marginally lower directional semantic alignment to the citing paper than non-cited candidates. When citations occur despite weak semantic alignment, they tend to be temporally proximate, though they do not exhibit a consistent prominence pattern. We conclude that under leakage-controlled conditions, citation formation exhibits substantial domain-specific heterogeneity, but chronological distance factors consistently contribute more predictive signal than directional semantic alignment or citation-network prominence factors.
Checked 2026-11-18 · 1 reference not checked yet
Overview
CoChat Summary
TL;DR
This paper introduces a leakage-mitigated framework for citation link prediction and applies it to ten Dimensions corpora (1975–2024), decomposing citation formation into semantic alignment (S), chronological distance (T), and citation-network prominence (Nprom). Under temporally rigorous evaluation, chronological recency dominates prediction across almost all corpora, while directional semantic alignment adds little beyond T and Nprom contributes negligibly once T is known.
Key findings
- Gradient Boosting achieves ROC-AUC from 0.530 (Corrosion Protection) to 0.734 (CRISPR) under temporally rigorous evaluation; temporal hold-out AUC is generally consistent with CV AUC, e.g. CRISPR 0.798, Protein Folding 0.745, Corrosion Protection 0.615.
- In the S-T-Nprom decomposition, T alone is the strongest single predictor in all ten corpora, with AUC(T) ranging from 0.577 (Memory Studies) to 0.824 (Protein Folding); S alone is weak, AUC(S) 0.490–0.536; Nprom alone is near-chance in most corpora. ΔT > 0 in all ten corpora, ΔS < 0 in all ten, and ΔN|ST is near zero or negative in eight of ten, with only CRISPR (+0.004) and Memory Studies (+0.005) showing marginal positive Nprom contribution beyond S+T.
- In four of ten corpora, cited papers exhibit marginally lower directional semantic alignment to the citing paper than non-cited candidates (mean difference < 0.01 on a 0–1 scale). When citations occur despite weak semantic alignment, they tend to be temporally proximate (ΔT < 0 in nine of ten corpora), with no consistent prominence pattern: seven of ten corpora show low-S citations as less prominent, while Income Inequality (+0.252) and Memory Studies (+0.094) show the opposite.
- Top-k ranking on CRISPR degrades as the candidate pool widens: NDCG@10 is approximately 0.79 at k = 10 and 0.30 at k = 100; MRR is 0.7340 at k = 10, 0.5102 at k = 50, and 0.2241 at k = 100.
Methods
- Design: a leakage-mitigated citation link prediction framework formalizing four temporal leakage types (future citation, future topology, metadata, semantic leakage); all network-derived features are computed from the temporally capped graph G_{t_A−1}, and negatives come from a constrained candidate space C(A) of non-cited papers within ±3 years of the true cited paper and in the same topical corpus, yielding balanced 1:1 positive-negative pairs.
- Data: ten Dimensions.ai corpora spanning Science (Protein Folding, CRISPR), Engineering (Additive Manufacturing, Corrosion Protection), BioMed (Neuroblastoma, Osteosarcoma), Social Science (Income Inequality, Organizational Behavior), and Humanities (Film Studies, Memory Studies); journal articles 1975–2024; record counts range from 26,486 (Corrosion Protection) to 393,342 (Film Studies), and valid pair counts range from 112 (Protein Folding) to 754 (Film Studies).
- Analysis: trained Logistic Regression, Linear SVM, Random Forest, and Gradient Boosting; used 5-fold stratified cross-validation, temporal hold-out (train ≤ 2015, test 2016–2020), rolling temporal windows, seven S/T/Nprom sub-models, SHAP and leave-one-out ablation, and top-k ranking with k ∈ {10, 50, 100} using MRR, NDCG@k, and Precision@k, with 95% bootstrap confidence intervals from 1000 resamples.
Limitations
- Paper states: SBERT pre-training introduces systemic semantic leakage that cannot be perfectly mitigated without chronologically re-trained language models; top-k experiments approximate retrieval realism over constrained candidate pools rather than full-corpus recommendation; pair counts are small for sparse domains (e.g., Corrosion Protection, Income Inequality), with cross-validation standard deviations of ±0.018 and ±0.014; the study relies on Gradient Boosting over engineered features rather than deep learning; and the ten datasets are single keyword-based queries, not complete disciplinary corpora, so conclusions are conditional on the keyword-defined topical corpus and ±3-year hard-negative candidate space.
- Paper states: co-citation overlap is typically zero because the citing paper A is newly published at t_A unless it previously existed as a preprint, so co-citation is retained as a structural baseline but contributes negligible predictive signal; D1 group sizes for semantically surprising citations are small in some corpora (n = 6–11), so corpus-specific deviations should not be over-interpreted.
- Evident from the text: the candidate-generation protocol restricts both topical and chronological distance variation, leaving relatively little semantic variation to discriminate cited from non-cited papers; the paper offers this as a plausible explanation for the negative ΔS result and notes that the S-T-Nprom profiles should not be interpreted as direct causal models or generalized beyond the constrained evaluation setting.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures











Verification
Claims the paper makes
0 Valid · 2 Not valid · 15 Not checked · 170 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The datasets exhibit expected variations in citation density and clustering across domains.”
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > 3.2 The Ten Citation Graphs
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The predictability of citation behavior varies substantially across domains.”
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > 5.2 Pairwise Classification Performance: Ten-Domain Comparison
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“in four of the ten corpora, cited papers exhibit marginally lower directional semantic alignment to the citing paper than non-cited candidates.”
- Also stated in: § 1 Introduction · plain
“In four of the ten corpora, cited papers exhibit marginally lower directional semantic alignment to the citing paper than non-cited candidates.”
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 Introduction · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When citations occur despite weak semantic alignment, they tend to be temporally proximate, though they do not exhibit a consistent prominence pattern.”
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > Abstract
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Under this decomposition, chronological recency (T) dominates prediction across almost all corpora.”
- Also stated in: § 1 Introduction · plain
“Our principal empirical finding is that chronological recency (T) dominates prediction across almost all corpora.”
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > Abstract
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 Introduction · plain
72 more claims
Create a free account to see all 187 claims and to verify the 12 references still open.
- It may state two separate claims as one. 71
- Probably not a substantial claim. 49
- The quote only partly states this claim. 2
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > 5.7 S-T-N prom Decomposition: Seven-Model Analysis
“The T-N prom collinearity diagnostics confirm that T and N prom are largely independent in most corpora (Pearson r ≈ 0.01-0.09),so the near-zero ∆ N |ST is not an artifact of multicollinearity: citation-network prominence genuinely adds nothing beyond chronological recency in the majority of corpora.”
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > 6.5 Limitations and Future Work
“it demonstrates that within keyword-defined corpora under temporal constraints, citation-network prominence is largely a chronological distance proxy rather than an independent structural signal.”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > 5.7 S-T-N prom Decomposition: Seven-Model Analysis
“T alone is the strongest single predictor in all ten corpora, with AUC(T) ranging from 0.577 (Memory Studies) to 0.824 (Protein Folding).”
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > 5.10 Estimated Logistic Effects of S and T on Citation Probability
“Income Inequality is the notable exception: the T panel is completely flat, indicating that chronological recency provides no discriminative signal in this corpus.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > 4.1 Mathematical Formalization
“All features associated with the candidate pair (A, B) are computed exclusively from G t A -1 .”
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > Semantic Similarity (S): Directional Semantic Similarity.
“The Kotlerman-Dagan Average Precision directional semantic alignment between the concatenated title and abstract of A and B, computed using SBERT (all-MiniLM-L6-v2) embeddings applied to the title and abstract fields available in Dimensions.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > Common References.
“Co-citation Overlap.The number of papers in G t A -1 citing both A and B, operationalizing co-citation (Small, 1973).Because A is newly published at time t A , this quantity is typically zero unless A previously existed as a preprint.”
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > 4.1 Mathematical Formalization
“V <t A = {v ∈ V | t v < t A }”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > 5.3 Rolling Temporal Evaluation: All Ten Domains
“Performance remains stable across all chronological distance windows for which sufficient data is available.”
§ Semantic Alignment, Chronological Distance, and Citation-Network Prominence in Citation Formation: Evidence from Ten Citation Corpora > 5.3 Rolling Temporal Evaluation: All Ten Domains
“By contrast, Protein Folding shows a volatile pattern (0.747 in W1, dropping to 0.469 in W4), and Memory Studies rises then collapses (0.682 in W2, 0.498 in W4)”
1 more finding
Create a free account to see every finding of this check.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Emergence of Scaling in Random Networks
Barabási, A.-L., & Albert, R. (1999) · Science · doi:10.1126/science.286.5439.509 · Cited by 36,135
Co‐citation in the scientific literature: A new measure of the relationship between two documents
Small, H. (1973) · Journal of the American Society for Information Science · doi:10.1002/asi.4630240406 · Cited by 5,345
Cumulated gain-based evaluation of IR techniques
Järvelin, K., & Kekäläinen, J. (2002) · ACM Transactions on Information Systems · doi:10.1145/582415.582418 · Cited by 4,841
Gradient boosting machines, a tutorial
Natekin, A., & Knoll, A. (2013) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2013.00021 · Cited by 3,887
Price, D. J. D. S. (1965) · Science · doi:10.1126/science.149.3683.510 · Cited by 3,122
The link‐prediction problem for social networks
Liben‐Nowell, D., & Kleinberg, J. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20591 · Cited by 3,043
Bibliographic coupling between scientific papers
Kessler, M. M. (1963) · American Documentation · doi:10.1002/asi.5090140103 · Cited by 2,855
Coauthorship networks and patterns of scientific collaboration
Newman, M. E. J. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0307545100 · Cited by 1,931
Learning to Rank for Information Retrieval
Liu, T.-Y. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000016 · Cited by 1,568
Fortunato, S., Bergstrom, C. T., Börner, K., Evans, J. A., Helbing, D., Milojević, S., Petersen, A. M., Radicchi, F., Sinatra, R., Uzzi, B., et al. (2018) · Science · doi:10.1126/science.aao0185 · Cited by 1,497
Research-paper recommender systems: a literature survey
Beel, J., Gipp, B., Langer, S., & Breitinger, C. (2016) · International Journal on Digital Libraries · doi:10.1007/s00799-015-0156-0 · Cited by 823
Quantifying Long-Term Scientific Impact
Wang, D., Song, C., & Barabási, A.-L. (2013) · Science · doi:10.1126/science.1237825 · Cited by 753
The anatomy of a large-scale social search engine
Horowitz, D., & Kamvar, S. D. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772735 · Cited by 425
Voorhees, E. M., & Harman, D. K. (2005) · The MIT Press eBooks · Cited by 392
Dimensions: Building Context for Search and Evaluation
Hook, D. W., Porter, S. J., & Herzog, C. (2018) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2018.00023 · Cited by 325
Co-authorship networks: a review of the literature
Kumar, S. (2015) · Aslib Journal of Information Management · doi:10.1108/ajim-09-2014-0116 · Cited by 247
Content-Based Citation Recommendation
Bhagavatula, C., Feldman, S., Power, R., & Ammar, W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1022 · Cited by 163
Towards a stratified learning approach to predict future citation counts
Chakraborty, T., Kumar, S., Goyal, P., Ganguly, N., & Mukherjee, A. (2014) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl.2014.6970190 · Cited by 106
Link prediction in citation networks
Shibata, N., Kajikawa, Y., & Sakata, I. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21664 · Cited by 104
Completeness degree of publication metadata in eight free-access scholarly databases
Delgado‐Quirós, L., & Ortega, J. L. (2024) · Quantitative Science Studies · doi:10.1162/qss_a_00286 · Cited by 99
Coauthorship and citation patterns in the Physical Review
Martin, T., Ball, B., Karrer, B., & Newman, M. E. J. (2013) · Physical Review E · doi:10.1103/physreve.88.012814 · Cited by 90
Nguyen, B. X., Luczak–Roesch, M., Dinneen, J. D., & Larivière, V. (2022) · Quantitative Science Studies · doi:10.1162/qss_a_00211 · Cited by 20
Books' Interest Grading and Fiction Readers' Search Actions During Query Reformulation Intervals
Mikkonen, A., & Vakkari, P. (2015) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/2756406.2756922 · Cited by 10
Citation proximus: The role of social and semantic ties on citations
Kozlowski, D., Pradier, C., Benz, P., Shokida, N. S., Andersen, J. P., & Larivière, V. (2025) · PLOS One · doi:10.1371/journal.pone.0335366 · Cited by 5
Citation proximus: the role of social and semantic ties in citing behaviour
Kozlowski, D., Pradier, C., Benz, P., Shokida, N., Andersen, J. P., & Larivière, V. (2025) · arXiv · doi:10.48550/arxiv.2502.13934
Research works printed but not linked: 4
- Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. Introduction to Informa- tion Retrieval. Cambridge University Press, 2008.
- Robert K. Merton. The Matthew Effect in Science. Science, 159(3810):56-63, 1968. doi: 10.1126/science.159.3810. · doi:10.1126/science.159.3810
- David M. W. Powers. Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness and Correlation. Journal of Machine Learning Technologies, 2(1):37-63, 2011.
- Ellen M. Voorhees and Donna K. Harman. TREC: Experiment and Evaluation in Information Retrieval. MIT Press, 2005.
Cited by
Still being checked.
Metadata
- Authors
- Boudourides, M.
- Affiliations
- Northwestern University (US)
- Corresponding author
- Boudourides, Moses
- ORCID
- Boudourides, Moses
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2026
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W7169606375
- Cited by
- 0
- References
- 22
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- SSRN Electronic Journal (submittedVersion, cc-by)
- Research Square (submittedVersion, cc-by)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
- Keywords
- temporal leakage, semantic alignment, citation networks, gradient boosting
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.