The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index; none of the 26 sources its claims cite is checked for support yet.
SPECTER: Document-level Representation Learning using Citation-informed Transformers
Open accessAbstract
Representation learning is a critical ingredient for natural language processing systems.Recent Transformer language models like BERT learn powerful textual representations, but these models are targeted towards token-and sentence-level training objectives and do not leverage information on inter-document relatedness, which limits their document-level representation power.For applications on scientific documents, such as classification and recommendation, the embeddings power strong performance on end tasks.We propose SPECTER, a new method to generate document-level embedding of scientific documents based on pretraining a Transformer language model on a powerful signal of document-level relatedness: the citation graph.Unlike existing pretrained language models, SPECTER can be easily applied to downstream applications without task-specific fine-tuning.Additionally, to encourage further research on document-level models, we introduce SCIDOCS, a new evaluation benchmark consisting of seven document-level tasks ranging from citation prediction, to document classification and recommendation.We show that SPECTER outperforms a variety of competitive baselines on the benchmark.1
Overview
CoChat Summary
TL;DR
SPECTER learns document-level embeddings of scientific papers by pretraining a SciBERT Transformer on citation-informed triplets, using only title and abstract at inference. It introduces the SCIDOCS seven-task benchmark and reports an average score of 80.0 across tasks/metrics, a 3.1-point absolute improvement over the next-best baseline.
Key findings
- SPECTER achieves an average of 80.0 across all SCIDOCS tasks/metrics, 3.1 points absolute over the next-best baseline. On MeSH classification it reaches F1 86.4 (+2.3 over best baseline) and on MAG F1 82.0 (+1.5); on co-view MAP 83.8 and co-read MAP 84.5, improving over Citeomatic by 2.7 and 4.0 points; on co-citation nDCG 94.8, +2.3 over SGC; on recommendation nDCG 53.9.
- In an online A/B test with 4,113 clicks, the SPECTER-based ranker improved clickthrough rate over the baseline by 46.5%.
- Ablations show removing the abstract drops average performance to 68.1, adding authors drops it to 67.3, using no hard negatives drops it to 78.4, and initializing from BERT-Large instead of SciBERT drops it to 77.5. Fixed SPECTER embeddings also outperform SciBERT fine-tuned on task-specific signals (SPECTER average 80.0 vs. multitask fine-tuning 78.0).
- t-SNE clustering quality for SPECTER is homogeneity 0.41 and completeness 0.72, compared with SciBERT’s 0.19 and 0.63.
Methods
- SPECTER initializes from SciBERT, encodes the concatenated title and abstract, and uses the final [CLS] pooled output as the paper vector. It pretrains with a triplet margin loss on the citation graph: query paper, positive cited paper, and negative uncited paper, using L2 distance and margin m=1. Hard negatives are citations of citations; training uses 2 hard negatives and 3 easy negatives per query.
- Training data: a Semantic Scholar subset of about 146K query papers (~26.7M tokens) and 32K validation papers, yielding about 684K training triples and 145K validation triples. Training uses Adam with LR 2e-5, slanted triangular LR, 2 epochs, batch size 4 with gradient accumulation for effective batch size 32, on a single Titan V GPU.
- Evaluation: SCIDOCS covers seven tasks—MeSH classification (23K papers, 11 disease classes), MAG topic classification (25K papers, 19 level-1 classes), direct citation prediction (~30K total papers; 1K queries, up to 5 cited and 25 uncited candidates), co-citation (30K), co-views (~30K), co-reads (~30K), and recommendation (22K clickthrough events; 20K/1K/1K split; propensity-adjusted P@1 and nDCG). Baselines include SIF, SciBERT, Sent-BERT, Citeomatic, and SGC; classification uses a linear SVM, and recommendation uses a feedforward ranking network with ten features including cosine embedding distance.
Limitations
- Stated: SPECTER uses only title and abstract; adding full text could improve accuracy, but full text is often not freely available and Transformer memory limits would require new techniques. Venue metadata was not normalized, and author names were sparse/out-of-vocabulary; adding authors or venues did not help and authors hurt performance.
- Stated: On the recommendation task, differences between embedding methods are generally smaller because embeddings are used alongside several other informative features, limiting their impact on overall performance.
- Evident from the text: evaluation is limited to scientific documents and the released SCIDOCS tasks, with no evidence for other domains or languages; the online A/B test covers one production recommender and 4,113 clicks, so broader deployment evidence is not provided.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
0 Valid · 2 Not valid · 18 Not checked · 56 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the recommendations task, as opposed to previous experiments, the differences in method scores are generally smaller.”
§ SPECTER: Document-level Representation Learning using Citation-informed Transformers > 5 Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that these pretrained embeddings substantially outperform alternative off-theshelf embeddings.”
§ SPECTER: Document-level Representation Learning using Citation-informed Transformers > 3. Fasttext-Sum
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As we show in our experiments ( §6), including hard negatives results in more accurate embeddings compared to using random negatives alone.”
§ SPECTER: Document-level Representation Learning using Citation-informed Transformers > 2.4 Selecting Negative Distractors
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We achieve substantial improvements over the strongest of a wide variety of baselines, demonstrating the effectiveness of our model.”
§ SPECTER: Document-level Representation Learning using Citation-informed Transformers > 8 Conclusions and Future Work
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe that the classifier performance when trained on our representations is better than when trained on any other baseline.”
§ SPECTER: Document-level Representation Learning using Citation-informed Transformers > 5 Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
25 more claims
Create a free account to see all 76 claims and to verify the 19 references still open.
- It may state two separate claims as one. 22
- Probably not a substantial claim. 29
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Covered model families span content-and embedding-based methods (e.g.,TF-IDF [63], BM25 [59] , SPECTER/SciBERT [17] ), graph-and HIN/GNN-based methods (GraphSAGE, Graph Attention Networks (GATs), meta-path reasoning), and hybrid/LLM pipelines (prompting, re-ranking, RAG).”
From A Critical Review on Citation Recommendation Systems Through a Graph-Centric Approach (2026) · cites this paper as [17]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853
(1996) · Cited by 20,179
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
A Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978386 · Cited by 11,000
Inductive Representation Learning on Large Graphs
Hamilton, W. L., Ying, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1706.02216 · Cited by 10,639
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
Scikit-learn: Machine Learning in Python
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078195 · Cited by 8,539
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
Distributed Representations of Sentences and Documents
Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 4,934
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100
Spectral Networks and Locally Connected Networks on Graphs
Bruna, J., Zaremba, W., Szlam, A., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6203 · Cited by 3,983
Software Framework for Topic Modelling with Large Corpora
Rek, R. R. U., & Sojka, P. (2010) · doi:10.13140/2.1.2393.1847 · Cited by 3,647
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
SciBERT: A Pretrained Language Model for Scientific Text
Beltagy, I., Lo, K., & Cohan, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1371 · Cited by 3,218
Self-citationA Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. (2020) · UTS ePRESS (University of Technology Sydney) · Cited by 3,174
Accelerating t-SNE using tree-based algorithms
van der Maaten, L. (2014) · doi:10.5555/2627435.2697068 · Cited by 2,226
Collaborative Filtering Recommender Systems
Schafer, J. B., Frankowski, D., Herlocker, J., & Sen, S. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-72079-9_9 · Cited by 2,120
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049
V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure
Rosenberg, A., & Hirschberg, J. (2007) · Cited by 1,322
Kiros, R., Zhu, Y., Salakhutdinov, R., Zemel, R. S., Torralba, A., Urtasun, R., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06726 · Cited by 1,293
AllenNLP: A Deep Semantic Natural Language Processing Platform
Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N. F., Peters, M., Schmitz, M., & Zettlemoyer, L. (2018) · Proceedings of Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/w18-2501 · Cited by 1,252
Simplifying Graph Convolutional Networks
Wu, F., Zhang, T., Junior, A. H. D. S., Fifty, C., Yu, T., & Weinberger, K. Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07153 · Cited by 1,165
(2017) · International Conference on Learning Representations · Cited by 1,075
Medical Subject Headings (MeSH).
(2000) · Europe PMC (PubMed Central) · Cited by 789
An Overview of Microsoft Academic Service (MAS) and Applications
Sinha, A., Shen, Z., Song, Y., Ma, H., Eide, D., Hsu, B.-J. (., & Wang, K. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2740908.2742839 · Cited by 658
An Empirical Evaluation of doc2vec with Practical Insights into Document Embedding Generation
Lau, J. H., & Baldwin, T. (2016) · Proceedings of the 1st Workshop on Representation Learning for NLP · doi:10.18653/v1/w16-1609 · Cited by 632
Construction of the Literature Graph in Semantic Scholar
Ammar, W., Groeneveld, D., Bhagavatula, C., Beltagy, I., Crawford, M., Downey, D., Dunkelberger, J., Elgohary, A., Feldman, S., Ha, V., et al. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers) · doi:10.18653/v1/n18-3011 · Cited by 358
Self-citationCANE: Context-Aware Network Embedding for Relation Modeling
Tu, C., Liu, H., Liu, Z., & Sun, M. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1158 · Cited by 243
A context-aware citation recommendation model with BERT and graph convolutional networks
Jeong, C., Jang, S., Park, E., & Choi, S. (2020) · Scientometrics · doi:10.1007/s11192-020-03561-y · Cited by 186
From Neural Re-Ranking to Neural Ranking: Learning a Sparse Representation for Inverted Indexing
Zamani, H., Dehghani, M., Croft, W. B., Learned-Miller, E., & Kamps, J. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3271800 · Cited by 185
Quantifying Sex Bias in Clinical Studies at Scale With Automated Data Extraction
Feldman, S., Ammar, W., Lo, K., Trepman, E., van Zuylen, M., & Etzioni, O. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.6700 · Cited by 177
Self-citationCohen, K. B., Johnson, H. L., Verspoor, K., Roeder, C., & Hunter, L. E. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-492 · Cited by 174
Content-Based Citation Recommendation
Bhagavatula, C., Feldman, S., Power, R., & Ammar, W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1022 · Cited by 163
Self-citationIdentifying a better measure of relatedness for mapping science
Klavans, R., & Boyack, K. W. (2006) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20274 · Cited by 158
Construction of the Literature Graph in Semantic ScholarPreprint
Ammar, W., Groeneveld, D., Bhagavatula, C., Beltagy, I., Crawford, M., Downey, D., Dunkelberger, J., Elgohary, A., Feldman, S., Ha, V., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.02262 · Cited by 132
Self-citationIs searching full text more effective than searching abstracts?
Lin, J. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-46 · Cited by 126
Word Mover’s Embedding: From Word2Vec to Document Embedding
Wu, L., Yen, I. E.-H., Xu, K., Xu, F., Balakrishnan, A., Chen, P., Ravikumar, P., & Witbrock, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1482 · Cited by 125
Distribution of information in biomedical abstracts and full-text publications
Schuemie, M. J., Weeber, M., Schijvenaars, B. J. A., van Mulligen, E. M., van der Eijk, C. C., Jelier, R., Mons, B., & Kors, J. A. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth291 · Cited by 123
DisSent: Learning Sentence Representations from Explicit Discourse Relations
Nie, A., Bennett, E., & Goodman, N. D. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1442 · Cited by 120
Neural Vector Spaces for Unsupervised Information Retrieval
Gysel, C. V., de Rijke, M., & Kanoulas, E. (2018) · ACM Transactions on Information Systems · doi:10.1145/3196826 · Cited by 107
Estimating Position Bias without Intrusive Interventions
Agarwal, A., Zaitsev, I., Wang, X., Li, C., Najork, M., & Joachims, T. (2019) · Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining · doi:10.1145/3289600.3291017 · Cited by 107
A Scalable Hybrid Research Paper Recommender System for Microsoft Academic
Kanakia, A., Shen, Z., Eide, D., & Wang, K. (2019) · The World Wide Web Conference · doi:10.1145/3308558.3313700 · Cited by 44
A Context-Aware Citation Recommendation Model with BERT and Graph Convolutional NetworksPreprint
Jeong, C., Jang, S., Shin, H., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.06464 · Cited by 29
Improved Semantic-Aware Network Embedding with Fine-Grained Word Alignment
Shen, D., Zhang, X., Henao, R., & Carin, L. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1209 · Cited by 27
From Neural Re-Ranking to Neural Ranking Learning a Sparse Representation for Inverted Indexing
Zamani, H., Dehghani, M., Bruce, C. W., Learned-Miller, E., & Kamps, J. (2018) · Data Archiving and Networked Services (DANS) · Cited by 26
Diffusion Maps for Textual Network EmbeddingPreprint
Zhang, X., Li, Y., Shen, D., & Carin, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09906 · Cited by 21
Research Paper Recommender Systems on Big Scholarly Data
Chen, T. T., & Lee, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-97289-3_20 · Cited by 14
Unsupervised Document Embedding With CNNsPreprint
Liu, C., Zhao, S., & Volkovs, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04168 · Cited by 9
Improving Textual Network Embedding with Global Attention via Optimal Transport
Chen, L., Wang, G., Tao, C., Shen, D., Cheng, P., Zhang, X., Wang, W., Zhang, Y., & Carin, L. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1512 · Cited by 9
Doc2Sent2Vec: A Novel Two-Phase Approach for Learning Document Representation
J, G., Gupta, M., & Varma, V. (2016) · Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval · doi:10.1145/2911451.2914717 · Cited by 7
A model of extended paragraph vector for document categorization and trend analysis
Liu, P., Wu, K. K., & Meng, H. M. L. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966147 · Cited by 3
Explaining Away Syntactic Structure in Semantic Document RepresentationsPreprint
Holmer, E., & Marfurt, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01620 · Cited by 1
Improving Textual Network Learning with Variational Homophilic Embeddings
Wang, W., Tao, C., Gan, Z., Wang, G., Chen, L., Zhang, X., Zhang, R., Yang, Q., Henao, R., & Carin, L. (2019) · arXiv · doi:10.48550/arxiv.1909.13456
A Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., et al. (2019) · arXiv · doi:10.48550/arxiv.1901.00596
Research works printed but not linked: 2
- William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017b. Inductive representation learning on large graphs. In NIPS.
- Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language under- standing by generative pre-training. arXiv.
Cited by
BioGPT: generative pre-trained transformer for biomedical text generation and mining
Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., & Liu, T.-Y. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac409 · Cited by 1,150
MTEB: Massive Text Embedding Benchmark
Muennighoff, N., Tazi, N., Magne, L., & Reimers, N. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.148 · Cited by 548
Large Language Models on Graphs: A Comprehensive Survey
Jin, B., Liu, G., Han, C., Jiang, M., Ji, H., & Han, J. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3469578 · Cited by 428
One Embedder, Any Task: Instruction-Finetuned Text Embeddings
Su, H., Shi, W., Kasai, J., Wang, Y., Hu, Y., Mari Ostendorf, E., Yih, W.-T., Smith, N. A., Zettlemoyer, L., & Yu, T. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.71 · Cited by 404
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., & Zaharia, M. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.272 · Cited by 353
Metadata
- Authors
- Cohan, A., Feldman, S., Beltagy, I., Downey, D., & Weld, D.
- Affiliations
- Allen Institute for Artificial Intelligence (US)
- University of Washington (US)
- Venue
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Published
- 2020
- Pages
- 2270-2282
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W3016913119
- Cited by
- 1,108
- References
- 62
- Citations by year
- 2026: 364 · 2025: 244 · 2024: 178 · 2023: 157 · 2022: 113 · 2021: 44 · 2020: 8
- FWCI
- 21.99
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Zenodo (CERN European Organization for Nuclear Research) (cc-by)
- Funders
- National Science Foundation (1936940)
- University of Washington
- Office of Naval Research (N00014- 18-1-2193)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- citation networks, citation prediction, scientific documents, transformer language models, pre-training, document classification, recommendations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

