SPECTER: Document-level Representation Learning using Citation-informed Transformers
Open accessThe DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index; 1 of the 26 sources its claims cite were checked for support.
Abstract
Representation learning is a critical ingredient for natural language processing systems.Recent Transformer language models like BERT learn powerful textual representations, but these models are targeted towards token-and sentence-level training objectives and do not leverage information on inter-document relatedness, which limits their document-level representation power.For applications on scientific documents, such as classification and recommendation, the embeddings power strong performance on end tasks.We propose SPECTER, a new method to generate document-level embedding of scientific documents based on pretraining a Transformer language model on a powerful signal of document-level relatedness: the citation graph.Unlike existing pretrained language models, SPECTER can be easily applied to downstream applications without task-specific fine-tuning.Additionally, to encourage further research on document-level models, we introduce SCIDOCS, a new evaluation benchmark consisting of seven document-level tasks ranging from citation prediction, to document classification and recommendation.We show that SPECTER outperforms a variety of competitive baselines on the benchmark.1
Overview
CoChat Summary
TL;DR
SPECTER learns document-level embeddings of scientific papers by pretraining a SciBERT Transformer on citation-informed triplets, using only title and abstract at inference. It introduces the SCIDOCS seven-task benchmark and reports an average score of 80.0 across tasks/metrics, a 3.1-point absolute improvement over the next-best baseline.
Key findings
- SPECTER achieves an average of 80.0 across all SCIDOCS tasks/metrics, 3.1 points absolute over the next-best baseline. On MeSH classification it reaches F1 86.4 (+2.3 over best baseline) and on MAG F1 82.0 (+1.5); on co-view MAP 83.8 and co-read MAP 84.5, improving over Citeomatic by 2.7 and 4.0 points; on co-citation nDCG 94.8, +2.3 over SGC; on recommendation nDCG 53.9.
- In an online A/B test with 4,113 clicks, the SPECTER-based ranker improved clickthrough rate over the baseline by 46.5%.
- Ablations show removing the abstract drops average performance to 68.1, adding authors drops it to 67.3, using no hard negatives drops it to 78.4, and initializing from BERT-Large instead of SciBERT drops it to 77.5. Fixed SPECTER embeddings also outperform SciBERT fine-tuned on task-specific signals (SPECTER average 80.0 vs. multitask fine-tuning 78.0).
- t-SNE clustering quality for SPECTER is homogeneity 0.41 and completeness 0.72, compared with SciBERT’s 0.19 and 0.63.
Methods
- SPECTER initializes from SciBERT, encodes the concatenated title and abstract, and uses the final [CLS] pooled output as the paper vector. It pretrains with a triplet margin loss on the citation graph: query paper, positive cited paper, and negative uncited paper, using L2 distance and margin m=1. Hard negatives are citations of citations; training uses 2 hard negatives and 3 easy negatives per query.
- Training data: a Semantic Scholar subset of about 146K query papers (~26.7M tokens) and 32K validation papers, yielding about 684K training triples and 145K validation triples. Training uses Adam with LR 2e-5, slanted triangular LR, 2 epochs, batch size 4 with gradient accumulation for effective batch size 32, on a single Titan V GPU.
- Evaluation: SCIDOCS covers seven tasks—MeSH classification (23K papers, 11 disease classes), MAG topic classification (25K papers, 19 level-1 classes), direct citation prediction (~30K total papers; 1K queries, up to 5 cited and 25 uncited candidates), co-citation (30K), co-views (~30K), co-reads (~30K), and recommendation (22K clickthrough events; 20K/1K/1K split; propensity-adjusted P@1 and nDCG). Baselines include SIF, SciBERT, Sent-BERT, Citeomatic, and SGC; classification uses a linear SVM, and recommendation uses a feedforward ranking network with ten features including cosine embedding distance.
Limitations
- Stated: SPECTER uses only title and abstract; adding full text could improve accuracy, but full text is often not freely available and Transformer memory limits would require new techniques. Venue metadata was not normalized, and author names were sparse/out-of-vocabulary; adding authors or venues did not help and authors hurt performance.
- Stated: On the recommendation task, differences between embedding methods are generally smaller because embeddings are used alongside several other informative features, limiting their impact on overall performance.
- Evident from the text: evaluation is limited to scientific documents and the released SCIDOCS tasks, with no evidence for other domains or languages; the online A/B test covers one production recommender and 4,113 clicks, so broader deployment evidence is not provided.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
1 Valid · 2 Not valid · 17 Not checked · 56 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“This is because SciBERT, like other pretrained language models, is trained via language modeling objectives, which only predict words or sentences given their in-document, nearby textual context.”
§ SPECTER: Document-level Representation Learning using Citation-informed Transformers > Document Representation
What the sources say
- Supportedcites Beltagy et al., 2019 matched to “SciBERT: A Pretrained Language Model for Scientific Text” (2019) source
The Methods (Sec. 2, Background) states that SciBERT follows BERT's architecture and pretraining tasks — 'predicting randomly masked tokens and predicting whether two sentences follow each other' — i.e. word-level and sentence-level objectives. The Corpus paragraph of Sec. 2 and Sec. 3.3 show these instances are built from the full text of papers with sentences split and packed up to 128/512 tokens, so the context the objectives use is nearby, in-document text. The Introduction likewise groups SciBERT with ELMo, GPT and BERT as unsupervised pretrained language models, covering the 'like other pretrained language models' clause. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the recommendations task, as opposed to previous experiments, the differences in method scores are generally smaller.”
§ SPECTER: Document-level Representation Learning using Citation-informed Transformers > 5 Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that these pretrained embeddings substantially outperform alternative off-theshelf embeddings.”
§ SPECTER: Document-level Representation Learning using Citation-informed Transformers > 3. Fasttext-Sum
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As we show in our experiments ( §6), including hard negatives results in more accurate embeddings compared to using random negatives alone.”
§ SPECTER: Document-level Representation Learning using Citation-informed Transformers > 2.4 Selecting Negative Distractors
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We achieve substantial improvements over the strongest of a wide variety of baselines, demonstrating the effectiveness of our model.”
§ SPECTER: Document-level Representation Learning using Citation-informed Transformers > 8 Conclusions and Future Work
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
25 more claims
Create a free account to see all 76 claims and to verify the 18 references still open.
- It may state two separate claims as one. 22
- Probably not a substantial claim. 29
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Covered model families span content-and embedding-based methods (e.g.,TF-IDF [63], BM25 [59] , SPECTER/SciBERT [17] ), graph-and HIN/GNN-based methods (GraphSAGE, Graph Attention Networks (GATs), meta-path reasoning), and hybrid/LLM pipelines (prompting, re-ranking, RAG).”
From A Critical Review on Citation Recommendation Systems Through a Graph-Centric Approach (2026) · cites this paper as [17]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853
(1996) · Cited by 20,179
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Research works printed but not linked: 2
- William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017b. Inductive representation learning on large graphs. In NIPS.
- Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language under- standing by generative pre-training. arXiv.
Cited by
Items 301–400 of 1,111
Yeganova, L. E., Kim, W. B., Tian, S., Comeau, D. C., Wilbur, W. J., & Lu, Z. (2025) · Nucleic Acids Research · doi:10.1093/nar/gkaf417 · Cited by 10
Distillation versus Contrastive Learning: How to Train Your Rerankers
Xu, Z., Huang, Z., Zhuang, S., & Srikumar, V. (2025) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2025.findings-ijcnlp.33 · Cited by 10
Detecting Cross-Lingual Information Gaps in Wikipedia
Ashrafimoghari, V. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543873.3587539 · Cited by 10
Word embeddings for retrieving tabular data from research publications
Berenguer, A., Mazón, J.-N., & Tomás, D. (2023) · Machine Learning · doi:10.1007/s10994-023-06472-0 · Cited by 10
An empirical study of the design choices for local citation recommendation systems
Medić, Z., & Šnajder, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116852 · Cited by 10
What Were People Searching For? A Query Log Analysis of An Academic Search Engine
Rohatgi, S., Giles, C. L., & Wu, J. F. (2021) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl52503.2021.00062 · Cited by 10
A General Framework for Producing Interpretable Semantic Text Embeddings
Sun, Y., Huang, Q., Tang, Y., Tung, A. K. H., & Yu, J. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2410.03435 · Cited by 10
CSPLADE: Learned Sparse Retrieval with Causal Language ModelsPreprint
Xu, Z., Feng, A., Tian, Y., Ding, H., & Cheong, L. L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.10816 · Cited by 11
Amiri, M., & Bocklitz, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.17277 · Cited by 11
Sparse Meets Dense: A Hybrid Approach to Enhance Scientific Document Retrieval
Mandikal, P., & Mooney, R. (2024) · arXiv · doi:10.48550/arxiv.2401.04055 · Cited by 9
Class-aware contrastive optimization for imbalanced text classification
Khvatskii, G., Moniz, N., Doan, K. D., & Chawla, N. V. (2025) · Discover Data · doi:10.1007/s44248-025-00064-0 · Cited by 9
MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch
Banar, N., Lotfi, E., Nooten, J. V., Arhiliuc, C., Kliocaite, M., & Daelemans, W. M. P. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.1236 · Cited by 9
Evolving from Rules to Learning in Urban Modeling and Planning Support Systems
Cai, Z. (2025) · Urban Science · doi:10.3390/urbansci9120508 · Cited by 9
Balancing the Blend: An Experimental Analysis of Trade-Offs in Hybrid Search
Wang, M., Tan, B., Gao, Y., Jin, H., Zhang, Y., Ke, X., Xu, X., & Zhu, Y. (2026) · Proceedings of the VLDB Endowment · doi:10.14778/3811243.3811246 · Cited by 9
Integrating Vector Databases across Embedding Models
Yang, B., Cao, Y., & Ren, Y. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3769803 · Cited by 9
Machine learning for semi-automated scoping reviews
Mozgai, S., Kaurloto, C., Winn, J. G., Leeds, A., Heylen, D., Hartholt, A., & Scherer, S. J. (2023) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2023.200249 · Cited by 9
Wang, Y., Qian, Y., Qi, X., Cao, N., & Wang, D. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3327387 · Cited by 9
Karakuzu, A., Boudreau, M., & Stikov, N. (2024) · Magnetic Resonance in Medical Sciences · doi:10.2463/mrms.rev.2023-0174 · Cited by 9
SciRus: Tiny and Powerful Multilingual Encoder for Scientific Texts
Gerasimenko, N. A., Vatolin, A., Ianina, A., & Vorontsov, K. V. (2024) · Doklady Mathematics · doi:10.1134/s1064562424602178 · Cited by 9
Brinner, M., & Zarrieß, S. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.1662 · Cited by 9
CSPLADE: Learned Sparse Retrieval with Causal Language Models
Xu, Z., Feng, A., Tian, Y., Ding, H., & Cheong, L. L. (2025) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2025.ijcnlp-long.7 · Cited by 9
PRM-KGED: paper recommender model using knowledge graph embedding and deep neural network
Thierry, N., Bao, B., Ali, Z., Tan, Z., Chatelain, I. B. C., & Kefalas, P. (2023) · Applied Intelligence · doi:10.1007/s10489-023-05162-7 · Cited by 9
Using neural networks to mine text and predict metabolic traits for thousands of microbes
Hackmann, T. J., & Zhang, B. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008757 · Cited by 9
Wu, L., Chen, S., Guo, L., Shpyleva, S., Harris, K., Fahmi, T., Flanigan, T., Tong, W., Xu, J., & Ren, Z. X. (2022) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2022.105287 · Cited by 9
Measuring the Commercial Potential of Science
Masclans, R., Hasan, S., & Cohen, W. M. (2024) · National Bureau of Economic Research · doi:10.3386/w32262 · Cited by 9
Scholar Inbox: Personalized Paper Recommendations for ScientistsPreprint
Flicke, M., Angrabeit, G., Iyengar, M., Protsenko, V., Shakun, I., Cicvaric, J., Kargi, B., He, H., Schuler, L., Scholz, L., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.08385 · Cited by 9
ChEmbed: Enhancing Chemical Literature Search Through Domain-Specific Text EmbeddingsPreprint
Kasmaee, A. S., Khodadad, M., Astaraki, M., Saloot, M. A., Sherck, N., Mahyar, H., & Samiee, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.01643 · Cited by 9
LGAI-EMBEDDING-Preview Technical ReportPreprint
Choi, J., Kim, H., Jang, H., Jun, C., Bae, K., Choi, H., Choi, S. J., Lee, H., & Yun, C. (2025) · arXiv.org · doi:10.48550/arxiv.2506.07438 · Cited by 2
Perplexity Trap: PLM-Based Retrievers Overrate Low Perplexity Documents
Wang, H., Dai, S., Zhao, H., Pang, L., Zhang, X., Wang, G., Dong, Z., Xu, J., & Wen, J.-R. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2503.08684 · Cited by 9
CSRv2: Unlocking Ultra-Sparse Embeddings
Guo, L., Wang, Y., Wen, T., Wang, Y., Feng, A., Chen, B., Jegelka, S., & You, C. (2026) · International Conference on Learning Representations · doi:10.48550/arxiv.2602.05735 · Cited by 9
Li, T., Wang, J., Zhang, Y., Li, S., & Chen, L. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736829 · Cited by 8
Multi-Facet Blending for Faceted Query-by-Example Retrieval
Do, H., Ryu, S., Kim, J., & Lee, G. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1388 · Cited by 8
Using language models to label clusters of scientific documents
Murray, D., Ni, C., Gu, W., & Hubbard, T. (2025) · Scientometrics · doi:10.1007/s11192-025-05445-5 · Cited by 8
BiBLoX realtime science trend mapping MLdriven forecasting
Kesgin, K., & Özer, D. Z. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00693-z · Cited by 8
Song, Z., Kong, X., Bao, X., Zhou, Y., Jiao, J., Liu, S., Zhou, Y., & Qi, H. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.131627 · Cited by 8
More Than Efficiency: Embedding Compression Improves Domain Adaptation in Dense Retrieval
Zuo, C., & Khashabi, D. (2026) · Workshop on Structured Understanding, Retrieval, and Generation in the LLM Era (SURGeLLM) · doi:10.18653/v1/2026.surgellm-1.24 · Cited by 8
Automating literature screening and curation with applications to computational neuroscience
Ji, Z., Guo, S., Qiao, Y., & McDougal, R. A. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae097 · Cited by 8
DoubleLingo: Causal Estimation with Large Language Models
Veljanovski, M., & Wood-Doughty, Z. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-short.71 · Cited by 8
Evaluation and Domain Adaptation of Similarity Models for Short Mathematical Texts
Steinfeldt, C., & Mihaljević, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-66997-2_14 · Cited by 8
CoSAEmb: Contrastive Section-aware Aspect Embeddings for Scientific Articles
Singh, S., & Singh, M. (2024) · Workshop on Scholarly Document Processing (SDP) · doi:10.18653/v1/2024.sdp-1.27 · Cited by 8
Jiawei, E., Zhang, Y., Xia, X., & Xu, X. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126599 · Cited by 8
An NLP-based method to mine gene and function relationships from published articles
Kumar, N., & Mukhtar, M. S. (2025) · Scientific Reports · doi:10.1038/s41598-025-91809-z · Cited by 8
RuSciBench: Open Benchmark for Russian and English Scientific Document Representations
Vatolin, A., Gerasimenko, N. A., Ianina, A., & Vorontsov, K. V. (2024) · Doklady Mathematics · doi:10.1134/s1064562424602191 · Cited by 8
QAEncoder: Towards Aligned Representation Learning in Question Answering Systems
Wang, Z., Yu, Q., Wei, S., Li, Z., Xiong, F., Wang, X., Niu, S., Liang, H., & Zhang, W. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.217 · Cited by 8
A Survey of Long-Document Retrieval in the PLM and LLM Era
Li, M., Zhang, Y., Lv, T., Zhao, S., Luo, M., Nie, E., & Zhou, G. (2026) · ACM Transactions on Information Systems · doi:10.1145/3849382 · Cited by 8
A multi-view method of scientific paper classification via heterogeneous graph embeddings
Lv, Y., Xie, Z. Y., Zuo, X., & Song, Y. (2022) · Scientometrics · doi:10.1007/s11192-022-04419-1 · Cited by 8
SciMine: An Efficient Systematic Prioritization Model Based on Richer Semantic Information
Guo, F., Luo, Y., Yang, L., & Zhang, Y. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591764 · Cited by 8
Query Generation with External Knowledge for Dense Retrieval
Cho, S., Jeong, S., Yang, W., & Park, J. C. (2022) · Workshop on Knowledge Extraction and Integration for Deep Learning Architectures: Deep Learning Inside Out (DeeLIO) · doi:10.18653/v1/2022.deelio-1.3 · Cited by 8
Improving the Generalizability of the Dense Passage Retriever Using Generated Datasets
Rajapakse, T. C., & de Rijke, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28238-6_7 · Cited by 8
Unsupervised Domain Adaptation for Sparse Retrieval by Filling Vocabulary and Word Frequency Gaps
Iida, H., & Okazaki, N. (2022) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2022.aacl-main.57 · Cited by 8
Meta-research on COVID-19: An overview of the early trendsPreprint
Colavizza, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02961 · Cited by 9
Learning Multiplex Representations on Text-Attributed Graphs with One Language Model EncoderPreprint
Jin, B., Zhang, W., Zhang, Y., Yu, M., Zhao, H., & Han, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.06684 · Cited by 8
Academic Article Recommendation Using Multiple PerspectivesPreprint
Church, K., Alonso, O., Vickers, P., Sun, J., Ebrahimi, A., & Chandrasekar, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.05836 · Cited by 8
Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to CollusionsPreprint
Hsieh, J.-Y., Raghunathan, A., & Shah, N. B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.06606 · Cited by 8
A Multidisciplinary Multimodal Aligned Dataset for Academic Data Processing
Song, H., Xu, H.-Y., Wang, Z.-K., Wang, Y.-F., & Li, J. (2025) · Scientific Data · doi:10.1038/s41597-025-04415-z · Cited by 4
AdaParse: An Adaptive Parallel PDF Parsing and Resource Scaling EnginePreprint
Siebenschuh, C., Hippe, K., Gökdemir, O., Brace, A., Khan, A., Hossain, K., Babuji, Y., Chia, N., Vishwanath, V., Stevens, R., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.01435 · Cited by 8
LightRetriever: A LLM-based Text Retrieval Architecture with Extremely Faster Query Inference
Ma, G., Ma, Y., Gou, X., Su, Z., Zhou, M., & Hu, S. (2026) · International Conference on Learning Representations · doi:10.48550/arxiv.2505.12260 · Cited by 8
Zhang, R. J., Kennard, N. N., Smith, D. S., McFarland, D. A., McCallum, A., & Keith, K. A. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.83 · Cited by 7
Relational Sentence Embedding for Flexible Semantic Matching
Wang, B., & Li, H. (2023) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2023.repl4nlp-1.20 · Cited by 7
Guedes, G. B., & da Silva, A. E. A. (2023) · doi:10.5121/csit.2023.131923 · Cited by 7
“Paper, Meet Code”: A Deep Learning Approach to Linking Scholarly Articles With GitHub Repositories
Puangjaktha, P., Choetkiertikul, M., & Tuarob, S. (2024) · IEEE Access · doi:10.1109/access.2024.3399767 · Cited by 7
Leveraging multiple control codes for aspect-controllable related paper recommendation
Long, K., Li, S., Tang, J., & Wang, T. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103879 · Cited by 7
SymTax: Symbiotic Relationship and Taxonomy Fusion for Effective Citation Recommendation
Goyal, K., Goel, M., Goyal, V., & Mohania, M. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.533 · Cited by 7
Predicting Scientific Impact Through Diffusion, Conformity, and Contribution Disentanglement
Xue, Z., He, G., Jiang, Z., Gu, S., Kang, Y., Zhao, S. X., & Lu, W. P. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679546 · Cited by 7
SciPrompt: Knowledge-augmented Prompting for Fine-grained Categorization of Scientific Topics
You, Z., Han, K., Zhu, H., Ludaescher, B., & Diesner, J. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.350 · Cited by 8
Dayarathne, R. N., Ranaweera, U., & Ganegoda, G. U. (2024) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-97-9255-9_26 · Cited by 7
Chatrinan, K., Noraset, T., & Tuarob, S. (2025) · Scientometrics · doi:10.1007/s11192-025-05233-1 · Cited by 7
LDIR: Low-Dimensional Dense and Interpretable Text Embeddings with Relative Representations
Wang, Y., Shen, Z., & Huang, H. F. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.743 · Cited by 7
CA-GAR: Context-Aware Alignment of LLM Generation for Document Retrieval
Yu, H., Kang, J., Li, R., Liu, Q., He, L., Huang, Z., Shen, S., & Lu, J. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.303 · Cited by 7
Jang, Y., Hong, S., Son, J., Park, S., Park, C., & Lim, H. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-srw.27 · Cited by 7
Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling
Zhang, H., Bi, K., Guo, J., Sun, X., Liu, S. B., Shi, D., Yin, D., & Cheng, X. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3839865 · Cited by 7
Structure and Semantics Preserving Document Representations
Raman, N., Shah, S., & Veloso, M. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3532062 · Cited by 7
New Siamese Neural Networks for Text Classification and Ontologies Alignment
Menad, S., Laddada, W., Abdeddaïm, S., & Soualmia, L. F. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44355-8_2 · Cited by 7
A Simple Angle-based Approach for Contrastive Learning of Unsupervised Sentence Representation
Jeong, Y. H., Han, M., & Chae, D. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.318 · Cited by 7
Tian, Y., Li, W., Wang, S., & Gu, Z. (2023) · Transactions in GIS · doi:10.1111/tgis.13059 · Cited by 7
MIReAD: Simple Method for Learning High-quality Representations from Scientific Documents
Razdaibiedina, A., & Brechalov, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.46 · Cited by 7
Research impact analysis of international funding agencies in the realm of allergy and immunology
Adachi, T., Ogawa, Y., Fukushi, T., Ito, K., Koizumi, A., Shirabe, M., Toriya, M., Hirako, J., Inomata, T., Masaki, K., et al. (2022) · Allergy · doi:10.1111/all.15249 · Cited by 7
NeuroCORD: A Language Model to Facilitate COVID-19-Associated Neurological Disorder Studies
Wu, L., Ali, S. D., Ali, H., Brock, T., Xu, J., & Tong, W. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19169974 · Cited by 7
Discovering Interdisciplinary Research Based on Neural Networks
He, T., Fu, W., Xu, J., Zhang, Z., Zhou, J., Yin, Y., & Xie, Z. (2022) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2022.908733 · Cited by 7
Large-scale Evaluation of Transformer-based Article Encoders on the Task of Citation RecommendationPreprint
Medić, Z., & Šnajder, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.05452 · Cited by 7
MultiSChuBERT: Effective Multimodal Fusion for Scholarly Document Quality PredictionPreprint
Wenniger, G. M. D. B., van Dongen, T., & Schomaker, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.07971 · Cited by 7
Tabatabaei, S. A., Fancher, S., Parsons, M., & Askari, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.05137 · Cited by 7
Park, S., Baek, J., Jeong, S., & Hwang, S. J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.10057 · Cited by 7
Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented GenerationPreprint
Zhang, H., Tang, M., Bi, K., Guo, J., Liu, S., Shi, D., Yin, D., & Cheng, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.05220 · Cited by 7
Distillation versus Contrastive Learning: How to Train Your RerankersPreprint
Xu, Z., Huang, Z., Zhuang, S., & Srikumar, V. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.08336 · Cited by 7
GeoGPT-RAG Technical ReportPreprint
Huang, F., Wu, F., Zhang, Z., Wang, Q., Zhang, L., Boquet, G. M., & Chen, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.09686 · Cited by 7
What Can Natural Language Processing Do for Peer Review?
Kuznetsov, I., Afzal, O. M., Dercksen, K., Dycke, N., Goldberg, A., Hope, T., Hovy, D., Kummerfeld, J. K., Lauscher, A., Leyton-Brown, K., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.06563 · Cited by 6
Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs
Negro, A., Montagna, F., Teng, M. N., Neal, T., Thomas, S., King, S., & Khan, R. (2023) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2023.1204801 · Cited by 6
Kim, H., Proctor, C. R., Walker, D., & McCarthy, R. R. (2023) · Journal of Medical Internet Research · doi:10.2196/42363 · Cited by 6
A data mining-based study on academic publication retractions in the 21st Century
Shen, Q., Gao, X., & Xiong, X. (2025) · Accountability in Research · doi:10.1080/08989621.2025.2528064 · Cited by 7
TITE: Token-Independent Text Encoder for Information Retrieval
Schlatt, F., Hagen, T., Potthast, M., & Hagen, M. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730094 · Cited by 6
CURE: A dataset for Clinical Understanding & Retrieval Evaluation
Sheikh, N. A., Marcos, D. B., Jousse, A.-L., Oladipo, A., Rousseau, O., & Lin, J. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737435 · Cited by 7
Uncovering drivers of climate research in policy with pretrained language models
Mahfouz, B., Capra, L., & Mulgan, G. (2025) · Patterns · doi:10.1016/j.patter.2025.101342 · Cited by 6
Shao, K., Liao, Z., Yu, J., Zhao, L., Li, Q., Huang, X., He, J., Tian, F., Zou, Y., Wang, X., et al. (2025) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1109/islped65674.2025.11261807 · Cited by 6
Context-Based URL Classification for Open Access Datasets and Software in Scholarly Documents
Salsabil, L., Obadage, R. R., Banerjee, B., Abeysinghe, Y., Alam, S., Färber, M., Ingram, W. A., Fox, E. A., & Wu, J. (2025) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl67857.2025.00031 · Cited by 6
Artificial Intelligence in Evidence-Based Medicine: On the Verge of Breakthrough
Nowak, A. J. (2022) · Springer eBooks · doi:10.1007/978-3-030-64573-1_43 · Cited by 6
Tracing the Retraction Cascade: Identifying Non-retracted but Potentially Retractable Articles
Usman, M., & Balke, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72437-4_7 · Cited by 6
Trippas, J. R. (2024) · ACM eBooks · doi:10.1145/3674127.3674135 · Cited by 6
Document Embeddings Enhance Biomedical Retrieval-Augmented Generation
Kong, Y., Yang, Z., Luo, L., Ding, Z., Wang, L., Liu, W., Zhang, Y., Xu, B., Wang, J., Sun, Y., et al. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822781 · Cited by 6
Benchmarking and Building Zero-Shot Hindi Retrieval Model with Hindi-BEIR and NLLB-E5
Acharya, A., Murthy, R., Kumar, V., & Sen, J. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.220 · Cited by 7
Metadata
- Authors
- Cohan, A., Feldman, S., Beltagy, I., Downey, D., & Weld, D.
- Affiliations
- Allen Institute for Artificial Intelligence (US)
- University of Washington (US)
- Venue
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Published
- 2020
- Pages
- 2270-2282
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W3016913119
- Cited by
- 1,108
- References
- 62
- Citations by year
- 2026: 364 · 2025: 244 · 2024: 178 · 2023: 157 · 2022: 113 · 2021: 44 · 2020: 8
- FWCI
- 21.99
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Zenodo (CERN European Organization for Nuclear Research) (cc-by)
- Funders
- National Science Foundation (1936940)
- University of Washington
- Office of Naval Research (N00014- 18-1-2193)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- citation networks, citation prediction, scientific documents, transformer language models, pre-training, document classification, recommendations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

