The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis
Open accessAbstract
During the last 3 years, a number of approaches for the normalization of RNA sequencing data have emerged in the literature, differing both in the type of bias adjustment and in the statistical strategy adopted. However, as data continue to accumulate, there has been no clear consensus on the appropriate normalization method to be used or the impact of a chosen method on the downstream analysis. In this work, we focus on a comprehensive comparison of seven recently proposed normalization methods for the differential analysis of RNA-seq data, with an emphasis on the use of varied real and simulated datasets involving different species and experimental designs to represent data characteristics commonly observed in practice. Based on this comparison study, we propose practical recommendations on the appropriate normalization method to be used and its impact on the differential analysis of RNA-seq data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome
Li, B., & Dewey, C. N. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-323 · Cited by 22,179
Full-length transcriptome assembly from RNA-Seq data without a reference genome
Grabherr, M. G., Haas, B. J., Yassour, M., Levin, J. Z., Thompson, D. A., Amit, I., Adiconis, X., Fan, L., Raychowdhury, R., Zeng, Q., et al. (2011) · Nature Biotechnology · doi:10.1038/nbt.1883 · Cited by 21,039
Differential expression analysis for sequence count data
Anders, S., & Huber, W. (2010) · Genome Biology · doi:10.1186/gb-2010-11-10-r106 · Cited by 15,581
Trapnell, C., Williams, B. A., Pertea, G., Mortazavi, A., Kwan, G., van Baren, M. J., Salzberg, S. L., Wold, B. J., & Pachter, L. (2010) · Nature Biotechnology · doi:10.1038/nbt.1621 · Cited by 15,556
Mapping and quantifying mammalian transcriptomes by RNA-Seq
Mortazavi, A., Williams, B. A., McCue, K., Schaeffer, L., & Wold, B. (2008) · Nature Methods · doi:10.1038/nmeth.1226 · Cited by 13,696
RNA-Seq: a revolutionary tool for transcriptomics
Wang, Z., Gerstein, M., & Snyder, M. (2009) · Nature Reviews Genetics · doi:10.1038/nrg2484 · Cited by 13,174
Gene expression profiling predicts clinical outcome of breast cancer
van 't Veer, L. J., Dai, H., van de Vijver, M. J., He, Y. D., Hart, A. A. M., Mao, M., Peterse, H. L., van der Kooy, K., Marton, M. J., Witteveen, A. T., et al. (2002) · Nature · doi:10.1038/415530a · Cited by 9,536
Bolstad, B. M., Irizarry, R. A., Åstrand, M., & Speed, T. P. (2003) · Bioinformatics · doi:10.1093/bioinformatics/19.2.185 · Cited by 8,266
A scaling normalization method for differential expression analysis of RNA-seq data
Robinson, M. D., & Oshlack, A. (2010) · Genome Biology · doi:10.1186/gb-2010-11-3-r25 · Cited by 7,876
Research works printed but not linked: 11
- 2010, 11, 909
- 2009, Nature; Physical Science (London), 4587234, 97
- 2010, 79, 709
- 2010, Cellular and molecular life sciences : CMLS, 674, 569
- 2010, Genome biology, 11106, R106
- Jaffrezic, 2007, Genetical research, 89, 19, 10.1017/S0016672307008646 · DOI
- 2003, Trends in genetics : TIG, 197, 362
- 2004, PNAS, 10116, 6062
- 2011, 10, 1
- 2002, Genome biology, 3, research0033, 10.1186/gb-2002-3-7-reports0033 · doi:10.1186/gb-2002-3-7-reports0033
- 2010, Genome biology, 112, R14
Cited by
jvenn: an interactive Venn diagram viewer
Bardou, P., Mariette, J., Escudié, F., Djemiel, C., & Klopp, C. (2014) · BMC Bioinformatics · doi:10.1186/1471-2105-15-293 · Cited by 2,489
Early allopolyploid evolution in the post-Neolithic Brassica napus oilseed genome
Chalhoub, B., Denoeud, F., Liu, S., Parkin, I. A. P., Tang, H., Wang, X., Chiquet, J., Belcram, H., Tong, C., Samans, B., et al. (2014) · Science · doi:10.1126/science.1253435 · Cited by 2,479
Current best practices in single-cell RNA-seq analysis: a tutorial
Luecken, M. D., & Theis, F. J. (2019) · Molecular Systems Biology · doi:10.15252/msb.20188746 · Cited by 2,231
Normalization of RNA-seq data using factor analysis of control genes or samples
Risso, D., Ngai, J., Speed, T. P., & Dudoit, S. (2014) · Nature Biotechnology · doi:10.1038/nbt.2931 · Cited by 2,052
Normalization and microbial differential abundance strategies depend upon data characteristics
Weiss, S., Xu, Z. Z., Peddada, S., Amir, A., Bittinger, K., Gonzalez, A., Lozupone, C., Zaneveld, J. R., Vázquez-Baeza, Y., Birmingham, A., et al. (2017) · Microbiome · doi:10.1186/s40168-017-0237-y · Cited by 2,051
RNA sequencing: the teenage years
Stark, R., Grzelak, M., & Hadfield, J. (2019) · Nature Reviews Genetics · doi:10.1038/s41576-019-0150-2 · Cited by 1,915
An overview of microRNAs: Biology, functions, therapeutics, and analysis methods
Saliminejad, K., Khorram Khorshid, H. R., Soleymani Fard, S., & Ghaffari, S. H. (2019) · Journal of Cellular Physiology · doi:10.1002/jcp.27486 · Cited by 1,872
Chong, J., Liu, P., Zhou, G., & Xia, J. (2020) · Nature Protocols · doi:10.1038/s41596-019-0264-1 · Cited by 1,702
Varet, H., Brillet-Guéguen, L., Coppée, J.-Y., & Dillies, M.-A. (2016) · PLOS ONE · doi:10.1371/journal.pone.0157022 · Cited by 1,346
Self-citationFernandes, A. D., Reid, J. N., Macklaim, J. M., McMurrough, T. A., Edgell, D. R., & Gloor, G. B. (2014) · Microbiome · doi:10.1186/2049-2618-2-15 · Cited by 1,329
Human housekeeping genes, revisited
Eisenberg, E., & Levanon, E. Y. (2013) · Trends in Genetics · doi:10.1016/j.tig.2013.05.010 · Cited by 1,282
SEQC/MAQC-III Consortium (2014) · Nature Biotechnology · doi:10.1038/nbt.2957 · Cited by 983
Gene co-expression analysis for functional classification and gene–disease predictions
van Dam, S., Võsa, U., van der Graaf, A., Franke, L., & de Magalhães, J. P. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbw139 · Cited by 995
Schurch, N. J., Schofield, P., Gierliński, M., Cole, C., Sherstnev, A., Singh, V., Wrobel, N., Gharbi, K., Simpson, G. G., Owen-Hughes, T., et al. (2016) · RNA · doi:10.1261/rna.053959.115 · Cited by 966
Characterization of the gut microbiome using 16S or shotgun metagenomics
Jovel, J., Patterson, J., Wang, W., Hotte, N., O'Keefe, S., Mitchel, T., Perry, T., Kao, D., Mason, A. L., Madsen, K. L., et al. (2016) · Frontiers in Microbiology · doi:10.3389/fmicb.2016.00459 · Cited by 942
A comparison of methods for differential expression analysis of RNA-seq data
Soneson, C., & Delorenzi, M. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-91 · Cited by 887
Individual intestinal symbionts induce a distinct population of RORγ + regulatory T cells
Sefik, E., Geva-Zatorsky, N., Oh, S., Konnikova, L., Zemmour, D., McGuire, A. M., Burzyn, D., Ortiz-Lopez, A., Lobera, M., Yang, J., et al. (2015) · Science · doi:10.1126/science.aaa9420 · Cited by 879
Data quality aware analysis of differential expression in RNA-seq with NOISeq R/Bioc package
Tarazona, S., Furió-Tarí, P., Turrà, D., Pietro, A. D., Nueda, M. J., Ferrer, A., & Conesa, A. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv711 · Cited by 779
Comprehensive evaluation of differential gene expression analysis methods for RNA-seq data
Rapaport, F., Khanin, R., Liang, Y., Pirun, M., Krek, A., Zumbo, P., Mason, C. E., Socci, N. D., & Betel, D. (2013) · Genome Biology · doi:10.1186/gb-2013-14-9-r95 · Cited by 733
TCC: an R package for comparing tag count data with robust normalization strategies
Sun, J., Nishiyama, T., Shimizu, K., & Kadota, K. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-219 · Cited by 628
Normalizing single-cell RNA sequencing data: challenges and opportunities
Vallejos, C. A., Risso, D., Scialdone, A., Dudoit, S., & Marioni, J. C. (2017) · Nature Methods · doi:10.1038/nmeth.4292 · Cited by 436
Zhao, Y., Li, M.-C., Konaté, M. M., Chen, L., Das, B., Karlovich, C., Williams, P. M., Evrard, Y. A., Doroshow, J. H., & McShane, L. M. (2021) · Journal of Translational Medicine · doi:10.1186/s12967-021-02936-w · Cited by 489
Comparison of software packages for detecting differential expression in RNA-seq studies
Seyednasrollah, F., Laiho, A., & Elo, L. L. (2015) · Briefings in Bioinformatics · doi:10.1093/bib/bbt086 · Cited by 421
“Additionally, we also consider the use of similarly preprocessed data in the analyses using the Trimmed Mean of M values (TMM) normalization [18], which was recently recommended in an extensive comparison of various normalization methods [19].”
MethodMisuse of RPKM or TPM normalization when comparing across samples and sequencing protocols
Zhao, S., Ye, Z., & Stanton, R. (2020) · RNA · doi:10.1261/rna.074922.120 · Cited by 460
Benchmarking of cell type deconvolution pipelines for transcriptomics data
Avila Cobos, F., Alquicira-Hernandez, J., Powell, J. E., Mestdagh, P., & De Preter, K. (2020) · Nature Communications · doi:10.1038/s41467-020-19015-1 · Cited by 446
Design and computational analysis of single-cell RNA-sequencing experiments
Bacher, R., & Kendziorski, C. (2016) · Genome Biology · doi:10.1186/s13059-016-0927-y · Cited by 440
Forreryd, A., Zeller, K. S., Lindberg, T., Johansson, H., & Lindstedt, M. (2016) · Toxicology in Vitro · doi:10.1016/j.tiv.2016.09.013 · Cited by 438
Burgos, K., Malenica, I., Metpally, R., Courtright, A., Rakela, B., Beach, T., Shill, H., Adler, C., Sabbagh, M., Villa, S., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0094839 · Cited by 385
Analysis of microbial compositions: a review of normalization and differential abundance analysis
Lin, H., & Peddada, S. D. (2020) · npj Biofilms and Microbiomes · doi:10.1038/s41522-020-00160-w · Cited by 341
“However, choosing the most effective quantile is nontrivial 21,42,44,47 – 49 .”
BackgroundSelecting between-sample RNA-Seq normalization methods from the perspective of their assumptions
Evans, C., Hardin, J., & Stoebel, D. M. (2018) · Briefings in Bioinformatics · doi:10.1093/bib/bbx008 · Cited by 374
Methods for normalizing microbiome data: An ecological perspective
McKnight, D. T., Huerlimann, R., Bower, D. S., Schwarzkopf, L., Alford, R. A., & Zenger, K. R. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13115 · Cited by 369
RNA-Seq Technology and Its Application in Fish Transcriptomics
Qian, X., Ba, Y., Zhuang, Q., & Zhong, G. (2014) · OMICS: A Journal of Integrative Biology · doi:10.1089/omi.2013.0110 · Cited by 350
Understanding sequencing data as compositions: an outlook and review
Quinn, T. P., Erb, I., Richardson, M. F., & Crowley, T. M. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty175 · Cited by 350
Identifying and mitigating bias in next-generation sequencing methods for chromatin biology
Meyer, C. A., & Liu, X. S. (2014) · Nature Reviews Genetics · doi:10.1038/nrg3788 · Cited by 338
Schurch, N., Schofield, P., Gierliński, M., Cole, C., Sherstnev, A., Singh, V., Wrobel, N., Gharbi, K., Simpson, G. G., Owen‐Hughes, T., et al. (2015) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.48550/arxiv.1505.02017 · Cited by 325
Cell fate inclination within 2-cell and 4-cell mouse embryos revealed by single-cell RNA sequencing
Biase, F. H., Cao, X., & Zhong, S. (2014) · Genome Research · doi:10.1101/gr.177725.114 · Cited by 313
Resolving host–pathogen interactions by dual RNA-seq
Westermann, A. J., Barquist, L., & Vogel, J. (2017) · PLOS Pathogens · doi:10.1371/journal.ppat.1006033 · Cited by 289
Transcriptome Profiling in Human Diseases: New Advances and Perspectives
Casamassimi, A., Federico, A., Rienzo, M., Esposito, S., & Ciccodicola, A. (2017) · International Journal of Molecular Sciences · doi:10.3390/ijms18081652 · Cited by 289
Balances: a New Perspective for Microbiome Analysis
Rivera-Pinto, J., Egozcue, J. J., Pawlowsky-Glahn, V., Paredes, R., Noguera-Julian, M., & Calle, M. L. (2018) · mSystems · doi:10.1128/msystems.00053-18 · Cited by 288
Data-based filtering for replicated high-throughput transcriptome sequencing experiments
Rau, A., Gallopin, M., Celeux, G., & Jaffrézic, F. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt350 · Cited by 272
Self-citationMicrobial ecology perturbation in human IgA deficiency
Fadlallah, J., El Kafsi, H., Sterlin, D., Juste, C., Parizot, C., Dorgham, K., Autaa, G., Gouas, D., Almeida, M., Lepage, P., et al. (2018) · Science Translational Medicine · doi:10.1126/scitranslmed.aan1217 · Cited by 272
pcaExplorer: an R/Bioconductor package for interacting with RNA-seq principal components
Marini, F., & Binder, H. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2879-1 · Cited by 271
Viral unmasking of cellular 5S rRNA pseudogene transcripts induces RIG-I-mediated immunity
Chiang, J. J., Sparrer, K. M. J., van Gent, M., Lässig, C., Huang, T., Osterrieder, N., Hopfner, K.-P., & Gack, M. U. (2018) · Nature Immunology · doi:10.1038/s41590-017-0005-y · Cited by 259
The Calcium Channel Subunit Alpha2delta2 Suppresses Axon Regeneration in the Adult CNS
Tedeschi, A., Dupraz, S., Laskowski, C. J., Xue, J., Ulas, T., Beyer, M., Schultze, J. L., & Bradke, F. (2016) · Neuron · doi:10.1016/j.neuron.2016.09.026 · Cited by 255
A broken promise: microbiome differential abundance methods do not control the false discovery rate
Hawinkel, S., Mattiello, F., Bijnens, L., & Thas, O. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbx104 · Cited by 199
Mor, M., Nardone, S., Sams, D. S., & Elliott, E. (2015) · Molecular Autism · doi:10.1186/s13229-015-0040-1 · Cited by 118
Peixoto, L., Risso, D., Poplawski, S. G., Wimmer, M. E., Speed, T. P., Wood, M. A., & Abel, T. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv736 · Cited by 105
GeneFriends: a human RNA-seq-based gene and transcript co-expression database
van Dam, S., Craig, T., & de Magalhães, J. P. (2015) · Nucleic Acids Research · doi:10.1093/nar/gku1042 · Cited by 115
Gunter, H. M., Fan, S., Xiong, F., Franchini, P., Fruciano, C., & Meyer, A. (2013) · Molecular Ecology · doi:10.1111/mec.12417 · Cited by 109
Zacher, B., Michel, M., Schwalb, B., Cramer, P., Tresch, A., & Gagneur, J. (2017) · PLOS ONE · doi:10.1371/journal.pone.0169249 · Cited by 91
“In order to make this approach applicable to sequencing data, we added a pseudocount and log-transformed the data before k-means clustering.”
MethodGeis-Asteggiante, L., Belew, A. T., Clements, V. K., Edwards, N. J., Ostrand-Rosenberg, S., El-Sayed, N. M., & Fenselau, C. (2018) · Journal of Proteome Research · doi:10.1021/acs.jproteome.7b00646 · Cited by 104
A reanalysis of mouse ENCODE comparative gene expression data
Gilad, Y., & Mizrahi-Man, O. (2015) · F1000Research · doi:10.12688/f1000research.6536.1 · Cited by 100
RNA‐seq Data: Challenges in and Recommendations for Experimental Design and Analysis
Williams, A. G., Thomas, S., Wyman, S. K., & Holloway, A. K. (2014) · Current Protocols in Human Genetics · doi:10.1002/0471142905.hg1113s83 · Cited by 93
Wright, C., Rajpurohit, A., Burke, E. E., Williams, C., Collado-Torres, L., Kimos, M., Brandon, N. J., Cross, A. J., Jaffe, A. E., Weinberger, D. R., et al. (2019) · BMC Genomics · doi:10.1186/s12864-019-5870-3 · Cited by 99
Molecular quantitative trait loci
Aguet, F., Alasoo, K., Li, Y. I., Battle, A., Im, H. K., Montgomery, S. B., & Lappalainen, T. (2023) · Nature Reviews Methods Primers · doi:10.1038/s43586-022-00188-6 · Cited by 98
Fusion-independent satellite cell communication to muscle fibers during load-induced hypertrophy
Murach, K. A., Vechetti, I. J., Van Pelt, D. W., Crow, S. E., Dungan, C. M., Figueiredo, V. C., Kosmac, K., Fu, X., Richards, C. I., Fry, C. S., et al. (2020) · Function · doi:10.1093/function/zqaa009 · Cited by 94
Genes, behavior and next‐generation RNA sequencing
Hitzemann, R., Bottomly, D., Darakjian, P., Walter, N., Iancu, O., Searles, R., Wilmot, B., & McWeeney, S. (2013) · Genes, Brain and Behavior · doi:10.1111/gbb.12007 · Cited by 88
“Normalization and weighting algorithms can be used to address these issues, but they in turn may introduce new biases (Bullard et al., 2010; Dillies et al. 2012).”
BackgroundROTS: reproducible RNA-seq biomarker detector—prognostic markers for clear cell renal cell cancer
Seyednasrollah, F., Rantanen, K., Jaakkola, P., & Elo, L. L. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv806 · Cited by 75
Bacteria differently regulate mRNA abundance to specifically respond to various stresses
Bartholomäus, A., Fedyunin, I., Feist, P., Sin, C., Zhang, G., Valleriani, A., & Ignatova, Z. (2016) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2015.0069 · Cited by 81
Enterobacter bugandensis: a novel enterobacterial species associated with severe clinical infection
Pati, N. B., Doijad, S. P., Schultze, T., Mannala, G. K., Yao, Y., Jaiswal, S., Ryan, D., Suar, M., Gwozdzinski, K., Bunk, B., et al. (2018) · Scientific Reports · doi:10.1038/s41598-018-23069-z · Cited by 80
RNA-Seq Reveals Novel Transcriptional Reorganization in Human Alcoholic Brain
Farris, S. P., & Dayne Mayfield, R. (2014) · International Review of Neurobiology · doi:10.1016/b978-0-12-801105-8.00011-4 · Cited by 49
Burcham, L. R., Le Breton, Y., Radin, J. N., Spencer, B. L., Deng, L., Hiron, A., Ransom, M. R., Mendonça, J. D. C., Belew, A. T., El-Sayed, N. M., et al. (2020) · mBio · doi:10.1128/mbio.02302-20 · Cited by 47
Leishman, E., Murphy, M., Mackie, K., & Bradshaw, H. B. (2018) · Biochimica et Biophysica Acta (BBA) - Molecular and Cell Biology of Lipids · doi:10.1016/j.bbalip.2018.02.001 · Cited by 42
Learning sparse log-ratios for high-throughput sequencing data
Gordon-Rodriguez, E., Quinn, T. P., & Cunningham, J. P. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab645 · Cited by 37
“Recently, a class of log-ratios known as balances (Egozcue and Pawlowsky-Glahn, 2005) have become of interest in microbiome applications, due to their interpretability as the relative weight between two sub-populations of bacteria (Morton et al. , 2019a; Quinn and Erb, 2019).”
BackgroundIonescu, R.-B., Nicaise, A. M., Reisz, J. A., Williams, E. C., Prasad, P., Willis, C. M., Simões-Abade, M. B. C., Sbarro, L., Dzieciatkowska, M., Stephenson, D., et al. (2024) · Cell Stem Cell · doi:10.1016/j.stem.2024.09.014 · Cited by 34
Bi, Y., & Davuluri, R. V. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-262 · Cited by 30
Essential guidelines for computational method benchmarking
Weber, L. M., Saelens, W., Cannoodt, R., Soneson, C., Hapfelmeier, A., Gardner, P. P., Boulesteix, A.-L., Saeys, Y., & Robinson, M. D. (2019) · Genome Biology · doi:10.1186/s13059-019-1738-8 · Cited by 21
Predicting chemotherapy response using a variational autoencoder approach
Wei, Q., & Ramsey, S. A. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04339-6 · Cited by 19
Lecaudey, L. A., Singh, P., Sturmbauer, C., Duenser, A., Gessl, W., & Ahi, E. P. (2021) · BMC Genomics · doi:10.1186/s12864-021-07775-z · Cited by 19
Quantifying Transcriptome Turnover on Phylogenies by Modeling Gene Expression as a Binary Trait
Thompson, A., May, M. R., Hopkins, B. R., Riedl, N., Barmina, O., Liebeskind, B. J., Zhao, L., Begun, D., & Kopp, A. (2025) · Molecular Biology and Evolution · doi:10.1093/molbev/msaf106 · Cited by 4
Metadata
- Authors
- Dillies, M.-A., Rau, A., Aubert, J., Hennequet-Antier, C., Jeanmougin, M., Servant, N., Keime, C., Marot, G., Castel, D., Estelle, J., Guernec, G., Jagla, B., Jouneau, L., Laloe, D., Le Gall, C., Schaeffer, B., Le Crom, S., Guedj, M., Jaffrezic, F., & on behalf of The French StatOmique Consortium
- Venue
- Briefings in Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2012-09-17
- Volume
- 14
- Issue
- 6
- Pages
- 671-683
- ISSN
- 1467-5463 (ISSN-L)
- 1477-4054
- OpenAlex
- W2165909549
- Cited by
- 1,298
- References
- 48
- FWCI
- 30.44
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.