The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.
Benchmarking Metagenomics Tools for Taxonomic Classification
Open accessAbstract
Summary Metagenomic sequencing is revolutionizing the detection and characterization of microbial species, and a wide variety of software tools are available to perform taxonomic classification of this data. The fast pace of development of these tools, alongside the complexity of metagenomic data, make it important that researchers are able to benchmark their performance. Here, we review current approaches for metagenomic analysis and evaluate the performance of 20 metagenomic classifiers using simulated and experimental datasets. We describe the key metrics used to assess performance, offer a framework for the comparison of additional classifiers, and discuss the future of metagenomic data analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Matplotlib: a 2D graphics environment
Hunter, J. D. (2007) · Computing in Science & Engineering · doi:10.1109/mcse.2007.55 · Cited by 39,249
The SILVA ribosomal RNA gene database project: improved data processing and web-based tools
Quast, C., Pruesse, E., Yilmaz, P., Gerken, J., Schweer, T., Yarza, P., Peplies, J., & Glöckner, F. O. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks1219 · Cited by 33,055
Fast and sensitive protein alignment using DIAMOND
Buchfink, B., Xie, C., & Huson, D. H. (2015) · Nature Methods · doi:10.1038/nmeth.3176 · Cited by 14,201
Data Structures for Statistical Computing in Python
McKinney, W. (2010) · Proceedings of the Python in Science Conference · doi:10.25080/majora-92bf1922-00a · Cited by 11,032
UniFrac: a New Phylogenetic Method for Comparing Microbial Communities
Lozupone, C., & Knight, R. (2005) · Applied and Environmental Microbiology · doi:10.1128/aem.71.12.8228-8235.2005 · Cited by 8,227
The relationship between Precision-Recall and ROC curves
Davis, J., & Goadrich, M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143874 · Cited by 6,559
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309
Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM
Li, H. (2013) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.48550/arxiv.1303.3997 · Cited by 5,159
Aligning sequence reads, clone sequences and assembly con*gs with BWA-MEM
Li, H. (2014) · arXiv (Cornell University) · doi:10.6084/m9.figshare.963153.v1 · Cited by 4,932
MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets
Steinegger, M., & Söding, J. (2017) · Nature Biotechnology · doi:10.1038/nbt.3988 · Cited by 4,840
Kraken: ultrafast metagenomic sequence classification using exact alignments
Wood, D. E., & Salzberg, S. L. (2014) · Genome Biology · doi:10.1186/gb-2014-15-3-r46 · Cited by 4,521
Davis, N. M., Proctor, D. M., Holmes, S. P., Relman, D. A., & Callahan, B. J. (2018) · Microbiome · doi:10.1186/s40168-018-0605-2 · Cited by 3,493
The Statistical Analysis of Compositional Data
Aitchison, J. (1982) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1982.tb01195.x · Cited by 3,405
Jupyter Notebooks – a publishing format for reproducible computational workflows
Thomas, K., Benjamin, R.-K., Fernando, P., Brian, G., Matthias, B., Jonathan, F., Kyle, K., Jessica, H., Jason, G., Sylvain, C., et al. (2016) · Positioning and Power in Academic Publishing: Players, Agents and Agendas · doi:10.3233/978-1-61499-649-1-87 · Cited by 3,063
Binning metagenomic contigs by coverage and composition
Alneberg, J., Bjarnason, B. S., de Bruijn, I., Schirmer, M., Quick, J., Ijaz, U. Z., Lahti, L., Loman, N. J., Andersson, A. F., & Quince, C. (2014) · Nature Methods · doi:10.1038/nmeth.3103 · Cited by 2,800
MaxBin 2.0: an automated binning algorithm to recover genomes from multiple metagenomic datasets
Wu, Y.-W., Simmons, B. A., & Singer, S. W. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btv638 · Cited by 2,793
Snakemake—a scalable bioinformatics workflow engine
Köster, J., & Rahmann, S. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts480 · Cited by 2,780
Dysfunction of the intestinal microbiome in inflammatory bowel disease and treatment
Morgan, X. C., Tickle, T. L., Sokol, H., Gevers, D., Devaney, K. L., Ward, D. V., Reyes, J. A., Shah, S. A., LeLeiko, N., Snapper, S. B., et al. (2012) · Genome Biology · doi:10.1186/gb-2012-13-9-r79 · Cited by 2,713
Yarza, P., Yilmaz, P., Pruesse, E., Glöckner, F. O., Ludwig, W., Schleifer, K.-H., Whitman, W. B., Euzéby, J., Amann, R., & Rosselló-Móra, R. (2014) · Nature Reviews Microbiology · doi:10.1038/nrmicro3330 · Cited by 2,589
Bracken: estimating species abundance in metagenomics data
(2016) · doi:10.7717/peerj-cs.104 · Cited by 2,333
MetaPhlAn2 for enhanced metagenomic taxonomic profiling
Truong, D. T., Franzosa, E. A., Tickle, T. L., Scholz, M., Weingart, G., Pasolli, E., Tett, A., Huttenhower, C., & Segata, N. (2015) · Nature Methods · doi:10.1038/nmeth.3589 · Cited by 2,185
Fast and sensitive taxonomic classification for metagenomics with Kaiju
Menzel, P., Ng, K. L., & Krogh, A. (2016) · Nature Communications · doi:10.1038/ncomms11257 · Cited by 2,118
Human gut microbes impact host serum metabolome and insulin sensitivity
Pedersen, H. K., Gudmundsdottir, V., Nielsen, H. B., Hyotylainen, T., Nielsen, T., Jensen, B. A. H., Forslund, K., Hildebrand, F., Prifti, E., Falony, G., et al. (2016) · Nature · doi:10.1038/nature18646 · Cited by 2,116
(2015) · doi:10.7717/peerj.1165 · Cited by 1,960
Bayesian community-wide culture-independent microbial source tracking
Knights, D., Kuczynski, J., Charlson, E. S., Zaneveld, J., Mozer, M. C., Collman, R. G., Bushman, F. D., Knight, R., & Kelley, S. T. (2011) · Nature Methods · doi:10.1038/nmeth.1650 · Cited by 1,856
Bioconda: sustainable and comprehensive software distribution for the life sciences
The Bioconda Team, Grüning, B., Dale, R., Sjödin, A., Chapman, B. A., Rowe, J., Tomkins-Tinch, C. H., Valieris, R., & Köster, J. (2018) · Nature Methods · doi:10.1038/s41592-018-0046-7 · Cited by 1,641
Chiu, C. Y., & Miller, S. A. (2019) · Nature Reviews Genetics · doi:10.1038/s41576-019-0113-7 · Cited by 1,620
Pasolli, E., Asnicar, F., Manara, S., Zolfo, M., Karcher, N., Armanini, F., Beghini, F., Manghi, P., Tett, A., Ghensi, P., et al. (2019) · Cell · doi:10.1016/j.cell.2019.01.001 · Cited by 1,591
Statistical Methods for Detecting Differentially Abundant Features in Clinical Metagenomic Samples
White, J. R., Nagarajan, N., & Pop, M. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000352 · Cited by 1,586
Centrifuge: rapid and sensitive classification of metagenomic sequences
Kim, D., Song, L., Breitwieser, F. P., & Salzberg, S. L. (2016) · Genome Research · doi:10.1101/gr.210641.116 · Cited by 1,438
A new genomic blueprint of the human gut microbiota
Almeida, A., Mitchell, A. L., Boland, M., Forster, S. C., Gloor, G. B., Tarkowska, A., Lawley, T. D., & Finn, R. D. (2019) · Nature · doi:10.1038/s41586-019-0965-1 · Cited by 1,314
Database indexing for production MegaBLAST searches
Morgulis, A., Coulouris, G., Raytselis, Y., Madden, T. L., Agarwala, R., & Schäffer, A. A. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btn322 · Cited by 1,299
Genomic surveillance elucidates Ebola virus origin and transmission during the 2014 outbreak
Gire, S. K., Goba, A., Andersen, K. G., Sealfon, R. S. G., Park, D. J., Kanneh, L., Jalloh, S., Momoh, M., Fullah, M., Dudas, G., et al. (2014) · Science · doi:10.1126/science.1259657 · Cited by 1,215
Self-citationVenkatesan, A., Tunkel, A. R., Bloch, K. C., Lauring, A. S., Sejvar, J., Bitnun, A., Stahl, J.-P., Mailles, A., Drebot, M., Rupprecht, C. E., et al. (2013) · Clinical Infectious Diseases · doi:10.1093/cid/cit458 · Cited by 1,197
Analyzing and minimizing PCR amplification bias in Illumina sequencing libraries
Aird, D., Ross, M. G., Chen, W.-S., Danielsson, M., Fennell, T., Russ, C., Jaffe, D. B., Nusbaum, C., & Gnirke, A. (2011) · Genome Biology · doi:10.1186/gb-2011-12-2-r18 · Cited by 1,174
Opportunistic data structures with applications
Ferragina, P., & Manzini, G. (2002) · Proceedings 41st Annual Symposium on Foundations of Computer Science · doi:10.1109/sfcs.2000.892127 · Cited by 1,063
Positioning and Power in Academic Publishing: Players, Agents and Agendas
Loizides, F., & Schmidt, B. (2016) · ORCA Online Research @Cardiff (Cardiff University) · Cited by 1,027
Benson, D. A. (2004) · Nucleic Acids Research · doi:10.1093/nar/gki063 · Cited by 1,016
Blauwkamp, T. A., Thair, S., Rosen, M. J., Blair, L., Lindner, M. S., Vilfan, I. D., Kawli, T., Christians, F. C., Venkatasubrahmanyam, S., Wall, G. D., et al. (2019) · Nature Microbiology · doi:10.1038/s41564-018-0349-6 · Cited by 962
Critical Assessment of Metagenome Interpretation—a benchmark of metagenomics software
Sczyrba, A., Hofmann, P., Belmann, P., Koslicki, D., Janssen, S., Dröge, J., Gregor, I., Majda, S., Fiedler, J., Dahms, E., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4458 · Cited by 870
Microbial strain-level population structure and genetic diversity from metagenomes
Truong, D. T., Tett, A., Pasolli, E., Huttenhower, C., & Segata, N. (2017) · Genome Research · doi:10.1101/gr.216242.116 · Cited by 839
New insights from uncultivated genomes of the global human gut microbiome
Nayfach, S., Shi, Z. J., Seshadri, R., Pollard, K. S., & Kyrpides, N. C. (2019) · Nature · doi:10.1038/s41586-019-1058-x · Cited by 737
Updating the 97% identity threshold for 16S ribosomal RNA OTUs
Edgar, R. C. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty113 · Cited by 723
Ounit, R., Wanamaker, S., Close, T. J., & Lonardi, S. (2015) · BMC Genomics · doi:10.1186/s12864-015-1419-2 · Cited by 632
A human gut bacterial genome and culture collection for improved metagenomic analyses
Forster, S. C., Kumar, N., Anonye, B. O., Almeida, A., Viciani, E., Stares, M. D., Dunn, M., Mkandawire, T. T., Zhu, A., Shao, Y., et al. (2019) · Nature Biotechnology · doi:10.1038/s41587-018-0009-7 · Cited by 580
Miller, S., Naccache, S. N., Samayoa, E., Messacar, K., Arevalo, S., Federman, S., Stryke, D., Pham, E., Fung, B., Bolosky, W. J., et al. (2019) · Genome Research · doi:10.1101/gr.238170.118 · Cited by 572
The GEM mapper: fast, accurate and versatile alignment by filtration
Marco-Sola, S., Sammeth, M., Guigó, R., & Ribeca, P. (2012) · Nature Methods · doi:10.1038/nmeth.2221 · Cited by 544
KrakenUniq: confident and fast metagenomics classification using unique k-mer counts
Breitwieser, F. P., Baker, D. N., & Salzberg, S. L. (2018) · Genome Biology · doi:10.1186/s13059-018-1568-0 · Cited by 474
Microbial abundance, activity and population genomic profiling with mOTUs2
Milanese, A., Mende, D. R., Paoli, L., Salazar, G., Ruscheweyh, H.-J., Cuenca, M., Hingamp, P., Alves, R., Costea, P. I., Coelho, L. P., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-08844-4 · Cited by 446
Consistent and correctable bias in metagenomic sequencing experiments
McLaren, M. R., Willis, A. D., & Callahan, B. J. (2019) · eLife · doi:10.7554/elife.46923 · Cited by 429
D’Amore, R., Ijaz, U. Z., Schirmer, M., Kenny, J. G., Gregory, R., Darby, A. C., Shakya, M., Podar, M., Quince, C., & Hall, N. (2016) · BMC Genomics · doi:10.1186/s12864-015-2194-9 · Cited by 405
Strain-level microbial epidemiology and population genomics from shotgun metagenomics
Scholz, M., Ward, D. V., Pasolli, E., Tolio, T., Zolfo, M., Asnicar, F., Truong, D. T., Tett, A., Morrow, A. L., & Segata, N. (2016) · Nature Methods · doi:10.1038/nmeth.3802 · Cited by 376
Gu, W., Crawford, E. D., O’Donovan, B. D., Wilson, M. R., Chow, E. D., Retallack, H., & DeRisi, J. L. (2016) · Genome Biology · doi:10.1186/s13059-016-0904-5 · Cited by 358
Viral Infections of the Lower Respiratory Tract: Old Viruses, New Viruses, and the Role of Diagnosis
Pavia, A. T. (2011) · Clinical Infectious Diseases · doi:10.1093/cid/cir043 · Cited by 354
Understanding sequencing data as compositions: an outlook and review
Quinn, T. P., Erb, I., Richardson, M. F., & Crowley, T. M. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty175 · Cited by 350
Use of simulated data sets to evaluate the fidelity of metagenomic processing methods
Mavromatis, K., Ivanova, N., Barry, K., Shapiro, H., Goltsman, E., McHardy, A. C., Rigoutsos, I., Salamov, A., Korzeniewski, F., Land, M., et al. (2007) · Nature Methods · doi:10.1038/nmeth1043 · Cited by 344
Comprehensive benchmarking and ensemble approaches for metagenomic classifiers
McIntyre, A. B. R., Ounit, R., Afshinnekoo, E., Prill, R. J., Hénaff, E., Alexander, N., Minot, S. S., Danko, D., Foox, J., Ahsanuddin, S., et al. (2017) · Genome Biology · doi:10.1186/s13059-017-1299-7 · Cited by 342
Loman, N. J., Constantinidou, C., Christner, M., Rohde, H., Chan, J. Z.-M., Quick, J., Weir, J. C., Quince, C., Smith, G. P., Betley, J. R., et al. (2013) · JAMA · doi:10.1001/jama.2013.3231 · Cited by 337
An evaluation of the accuracy and speed of metagenome analysis tools
Lindgreen, S., Adair, K. L., & Gardner, P. P. (2016) · Scientific Reports · doi:10.1038/srep19233 · Cited by 331
Metagenomics for pathogen detection in public health
Miller, R. R., Montoya, V., Gardy, J. L., Patrick, D. M., & Tang, P. (2013) · Genome Medicine · doi:10.1186/gm485 · Cited by 272
ConStrains identifies microbial strains in metagenomic datasets
Luo, C., Knight, R., Siljander, H., Knip, M., Xavier, R. J., & Gevers, D. (2015) · Nature Biotechnology · doi:10.1038/nbt.3319 · Cited by 267
Sinha, R., Stanley, G., Gulati, G. S., Ezran, C., Travaglini, K. J., Wei, E., Chan, C. K. F., Nabhan, A. N., Su, T., Morganti, R. M., et al. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/125724 · Cited by 209
Ross, E. M., Moate, P. J., Marett, L. C., Cocks, B., & Hayes, B. J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073056 · Cited by 181
Accurate read-based metagenome characterization using a hierarchical suite of unique signatures
Freitas, T. A. K., Li, P.-E., Scholz, M. B., & Chain, P. S. G. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv180 · Cited by 179
Proceedings 41st Annual Symposium on Foundations of Computer Science
(2000) · doi:10.1109/sfcs.2000 · Cited by 175
Nasko, D. J., Koren, S., Phillippy, A. M., & Treangen, T. J. (2018) · Genome Biology · doi:10.1186/s13059-018-1554-6 · Cited by 170
Walker, M., Pedamallu, C. S., Ojesina, A. I., Bullman, S., Sharpe, T., Whelan, C. W., & Meyerson, M. M. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty501 · Cited by 139
Higher classification sensitivity of short metagenomic reads with CLARK- S
Ounit, R., & Lonardi, S. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw542 · Cited by 136
Viral nucleic acids in human plasma pools
Zhang, W., Li, L., Deng, X., Blümel, J., Nübling, C. M., Hunfeld, A., Baylis, S. A., & Delwart, E. (2016) · Transfusion · doi:10.1111/trf.13692 · Cited by 127
Viral Diagnostics in Plants Using Next Generation Sequencing: Computational Analysis in Practice
Jones, S., Baizan-Edge, A., MacFarlane, S. A., & Torrance, L. (2017) · Frontiers in Plant Science · doi:10.3389/fpls.2017.01770 · Cited by 119
Somasekar, S., Lee, D., Rule, J. A., Naccache, S. N., Stone, M., Busch, M. P., Sanders, C. M., Lee, W. M., & Chiu, C. Y. (2017) · Clinical Infectious Diseases · doi:10.1093/cid/cix596 · Cited by 99
Towards precision quantification of contamination in metagenomic sequencing experiments
Zinter, M. S., Mayday, M. Y., Ryckman, K. K., Jelliffe‐Pawlowski, L. L., & DeRisi, J. L. (2019) · Microbiome · doi:10.1186/s40168-019-0678-6 · Cited by 89
Assessing taxonomic metagenome profilers with OPAL
Meyer, F., Bremges, A., Belmann, P., Janssen, S., McHardy, A. C., & Koslicki, D. (2019) · Genome Biology · doi:10.1186/s13059-019-1646-y · Cited by 80
Yang, Y., Jiang, X., & Zhang, T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0110947 · Cited by 60
MetaMeta: integrating metagenome analysis tools to improve taxonomic profiling
Piro, V. C., Matschkowski, M., & Renard, B. Y. (2017) · Microbiome · doi:10.1186/s40168-017-0318-y · Cited by 59
Ainsworth, D., Sternberg, M. J. E., Raczy, C., & Butcher, S. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkw1248 · Cited by 50
Shrinkage improves estimation of microbial associations under different normalization methods
Badri, M. H., Kurtz, Z. D., Bonneau, R., & Müller, C. L. (2020) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqaa100 · Cited by 43
Liu, X., Yu, Y., Liu, J., Elliott, C. F., Qian, C., & Liu, J. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx432 · Cited by 40
BLAST-based validation of metagenomic sequence assignments
Bazinet, A. L., Ondov, B., Sommer, D. D., & Ratnayake, S. (2018) · PeerJ · doi:10.7717/peerj.4892 · Cited by 37
Corvelo, A., Clarke, W. E., Robine, N., & Zody, M. C. (2018) · Genome Research · doi:10.1101/gr.225276.117 · Cited by 34
Using pseudoalignment and base quality to accurately quantify microbial community composition
Reppell, M., & Novembre, J. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006096 · Cited by 24
EnSVMB: Metagenomics Fragments Classification using Ensemble SVM and BLAST
Jiang, Y., Wang, J., Xia, D., & Yu, G. (2017) · Scientific Reports · doi:10.1038/s41598-017-09947-y · Cited by 11
Cited by
Improved metagenomic analysis with Kraken 2
Wood, D. E., Lu, J., & Langmead, B. (2019) · Genome Biology · doi:10.1186/s13059-019-1891-0 · Cited by 7,061
Beghini, F., McIver, L. J., Blanco-Míguez, A., Dubois, L., Asnicar, F., Maharjan, S., Mailyan, A., Manghi, P., Scholz, M., Thomas, A. M., et al. (2021) · eLife · doi:10.7554/elife.65088 · Cited by 2,303
A practical guide to amplicon and metagenomic analysis of microbiome data
Liu, Y.-X., Qin, Y., Chen, T., Lu, M., Qian, X., Guo, X., & Bai, Y. (2021) · Protein & Cell · doi:10.1007/s13238-020-00724-8 · Cited by 864
Metagenome analysis using the Kraken software suite
Lu, J., Rincon, N., Wood, D. E., Breitwieser, F. P., Pockrandt, C., Langmead, B., Salzberg, S. L., & Steinegger, M. (2022) · Nature Protocols · doi:10.1038/s41596-022-00738-y · Cited by 756
A pan-cancer mycobiome analysis reveals fungal involvement in gastrointestinal and lung tumors
Dohlman, A. B., Klug, J., Mesko, M., Gao, I. H., Lipkin, S. M., Shen, X., & Iliev, I. D. (2022) · Cell · doi:10.1016/j.cell.2022.09.015 · Cited by 413
Metadata
- Authors
- Ye, S. H., Siddle, K. J., Park, D. J., & Sabeti, P. C.
- Venue
- Cell
- Publisher
- Elsevier BV
- Published
- 2019-08
- Volume
- 178
- Issue
- 4
- Pages
- 779-794
- ISSN
- 0092-8674 (ISSN-L)
- OpenAlex
- W2968450569
- Cited by
- 593
- References
- 84
- FWCI
- 17.76
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.