On the Use of ArXiv as a Dataset
Open accessAbstract
The arXiv has collected 1.5 million pre-print articles over 28 years, hosting literature from scientific fields including Physics, Mathematics, and Computer Science. Each pre-print features text, figures, authors, citations, categories, and other metadata. These rich, multi-modal features, combined with the natural graph structure---created by citation, affiliation, and co-authorship---makes the arXiv an exciting candidate for benchmarking next-generation models. Here we take the first necessary steps toward this goal, by providing a pipeline which standardizes and simplifies access to the arXiv's publicly available data. We use this pipeline to extract and analyze a 6.7 million edge citation graph, with an 11 billion word corpus of full-text research articles. We present some baseline classification results, and motivate application of more exciting generative graph models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Non-trivial dataset overlap included s2orc, Arxiv, and the PubMed Central subset of the Pile.
“Non-trivial datasets overlap included s2orc (Lo et al., 2020), Arxiv (Clement et al., 2019) and the PubMed Central subset of the Pile (Gao et al., 2020).”
From The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset (2023) · cites this paper as Clement et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 18
- Akiko Aizawa, Michael Kohlhase, Iadh Ounis, and Moritz Schubotz. Ntcir-11 math-2 task overview. In NTCIR , volume 11, pp. 88-98. Citeseer, 2014.
- Alexander A. Alemi and Paul Ginsparg. Text segmentation based on semantic word embeddings. CoRR , abs/1503.05543, 2015. URL http://arxiv.org/abs/1503.05543 . · arXiv
- Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261 , 2018. URL https://arxiv.org/abs/1806.01261 . · arXiv
- Daniel Cer, Yinfei Yang, Sheng yi Kong, Nan Hua, Nicole Lyn Untalan Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cspedes, Steve Yuan, Chris Tar, Yun hsuan Sung, Brian Strope, and Ray Kurzweil. Universal sentence encoder. In In submission to: EMNLP demonstration , Brussels, Belgium, 2018. URL https://arxiv.org/abs/1803.11175 . In submission. · arXiv
- Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005 , 2013. URL https://arxiv.org/abs/1312.3005 . · arXiv
- Andrew M. Dai, Christopher Olah, and Quoc V. Le. Document embedding with paragraph vectors. CoRR , abs/1507.07998, 2015. URL http://arxiv.org/abs/1507.07998 . · arXiv
- Walter Dempsey, Brandon Oselio, and Alfred Hero. Hierarchical network models for structured exchangeable interaction processes. arXiv preprint arXiv:1901.09982 , 2019. · arXiv
- Steffen Eger, Chao Li, Florian Netzer, and Iryna Gurevych. Predicting research trends from arxiv. arXiv preprint arXiv:1903.02831 , 2019. · arXiv
- Johannes Gehrke, Paul Ginsparg, and Jon Kleinberg. Overview of the 2003 kdd cup. ACMSIGKDD Explorations Newsletter , 5(2):149-151, 2003.
- Palash Goyal and Emilio Ferrara. Graph embedding techniques, applications, and performance: A survey. Knowledge-Based Systems , 151:78-94, 2018. URL https://arxiv.org/abs/1705.02801 . · arXiv
- Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems , pp. 1024-1034, 2017a. URL https://arxiv.org/abs/1706.02216 . · arXiv
- William L Hamilton, Rex Ying, and Jure Leskovec. Representation learning on graphs: Methods and applications. 2017b. URL https://arxiv.org/abs/1709.05584 . · arXiv
- Carl Lagoze and Herbert Van de Sompel. The open archives initiative: Building a low-barrier interoperability framework. In Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries , pp. 54-62. ACM, 2001.
- David Liben-Nowell and Jon Kleinberg. The link-prediction problem for social networks. Journal of the American society for information science and technology , 58(7):1019-1031, 2007.
- WenyuanLiu, Stanisław Saganowski, Przemysław Kazienko, and Siew Ann Cheong. Using machine learning to predict the evolution of physics research. arXiv preprint arXiv:1810.12116 , 2018. · arXiv
- Michal Lopuszynski and Lukasz Bolikowski. Tagging scientific publications using wikipedia and natural language processing tools. comparison on the arxiv dataset. CoRR , abs/1309.0326, 2013. URL http://arxiv.org/abs/1309.0326 . · arXiv
- Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE , 104(1):11-33, 2016. URL https://arxiv.org/abs/1503.00759 . · arXiv
- Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV) , 115(3):211-252, 2015. doi: 10.1007/s11263-015-0816-y. · DOI · Paper page
Cited by
Sorted by citations
Can AI-Generated Text be Reliably Detected?
Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., & Feizi, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11156 · Cited by 161 · see citing works
“9 We use WebText and ArXiv abstracts [39] datasets as human text distributions.”
MethodThe BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Laurençon, H., Saulnier, L., Wang, T., Akiki, C., del Moral, A. V., Scao, T. L., Von Werra, L., Mou, C., Ponferrada, E. G., Nguyen, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03915 · Cited by 61 · see citing works
SciCap: Generating Captions for Scientific Figures
Hsu, T.-Y., Giles, C. L., & Huang, T.-H. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.277 · Cited by 59 · see citing works
- Authors
- Clement, C. B., Bierbaum, M., O'Keeffe, K. P., & Alemi, A. A.
- Affiliations
- Cornell University (US)
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1905.00075
- OpenAlex
- W2943043881
- Cited by
- 53
- References
- 0
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 3 · 2023: 14 · 2022: 8 · 2021: 18 · 2020: 3 · 2019: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
- Keywords
- arXiv, citation networks, graph generative models, data pipelines, multimodal data, co-authorship networks, text classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
