The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Normalization, testing, and false discovery rate estimation for RNA-sequencing data
Open accessAbstract
We discuss the identification of genes that are associated with an outcome in RNA sequencing and other sequence-based comparative genomic experiments. RNA-sequencing data take the form of counts, so models based on the Gaussian distribution are unsuitable. Moreover, normalization is challenging because different sequencing experiments may generate quite different total numbers of reads. To overcome these difficulties, we use a log-linear model with a new approach to normalization. We derive a novel procedure to estimate the false discovery rate (FDR). Our method can be applied to data with quantitative, two-class, or multiple-class outcomes, and the computation is fast even for large data sets. We study the accuracy of our approaches for significance calculation and FDR estimation, and we demonstrate that our method has potential advantages over existing methods that are based on a Poisson or negative binomial model. In summary, this work provides a pipeline for the significance analysis of sequencing data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Normalization methods that take this into account are TMM [47], DESeq [48], PoissonSeq [49] and UpperQuartile [45], which ignore highly variable and/or highly expressed features.”
From A survey of best practices for RNA-seq data analysis (2016) · cites this paper as [49]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Robinson, M. D., McCarthy, D. J., & Smyth, G. K. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btp616 · Cited by 41,364
Differential expression analysis for sequence count data
Anders, S., & Huber, W. (2010) · Genome Biology · doi:10.1186/gb-2010-11-10-r106 · Cited by 15,581
Mapping and quantifying mammalian transcriptomes by RNA-Seq
Mortazavi, A., Williams, B. A., McCue, K., Schaeffer, L., & Wold, B. (2008) · Nature Methods · doi:10.1038/nmeth.1226 · Cited by 13,696
RNA-Seq: a revolutionary tool for transcriptomics
Wang, Z., Gerstein, M., & Snyder, M. (2009) · Nature Reviews Genetics · doi:10.1038/nrg2484 · Cited by 13,174
In silico prediction of protein-protein interactions in human macrophages
Souiai, O., Guerfali, F. Z., Miled, S. B., Brun, C., & Benkahla, A. (2014) · BMC Research Notes · doi:10.1186/1756-0500-7-157 · Cited by 10,785
Significance analysis of microarrays applied to the ionizing radiation response
Tusher, V. G., Tibshirani, R., & Chu, G. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.091062498 · Cited by 10,568
Self-citationStatistical significance for genomewide studies
Storey, J. D., & Tibshirani, R. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1530509100 · Cited by 9,692
Self-citationHapeman, D. E., & Agresti, A. (1991) · Technometrics · doi:10.1080/00401706.1991.10484817 · Cited by 9,407
A scaling normalization method for differential expression analysis of RNA-seq data
Robinson, M. D., & Oshlack, A. (2010) · Genome Biology · doi:10.1186/gb-2010-11-3-r25 · Cited by 7,876
A Direct Approach to False Discovery Rates
Storey, J. D. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00346 · Cited by 5,603
Exploring the metabolic and genetic control of gene expression on a genomic scale.
Derisi, J., Iyer, V., & Brown, P. (1997) · Science · doi:10.1126/science.278.5338.680 · Cited by 4,832
Spellman, P. T., Sherlock, G. J., Zhang, M. Q., Iyer, V. R., Anders, K. R., Eisen, M. B., Brown, P. O., Botstein, D., & Futcher, B. (1998) · Molecular Biology of the Cell · doi:10.1091/mbc.9.12.3273 · Cited by 4,724
DEGseq: an R package for identifying differentially expressed genes from RNA-seq data
Wang, L., Feng, Z., Wang, X., Wang, X., & Zhang, X. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btp612 · Cited by 4,553
RNA-seq: An assessment of technical reproducibility and comparison with gene expression arrays
Marioni, J. C., Mason, C. E., Mane, S. M., Stephens, M., & Gilad, Y. (2008) · Genome Research · doi:10.1101/gr.079558.108 · Cited by 2,771
The Transcriptional Landscape of the Yeast Genome Defined by RNA Sequencing
Nagalakshmi, U., Wang, Z., Waern, K., Shou, C., Raha, D., Gerstein, M., & Snyder, M. (2008) · Science · doi:10.1126/science.1158441 · Cited by 2,479
Exploring the new world of the genome with DNA microarrays
Brown, P. O., & Botstein, D. (1999) · Nature Genetics · doi:10.1038/4462 · Cited by 2,430
The positive false discovery rate: a Bayesian interpretation and the q-value
Storey, J. D. (2003) · The Annals of Statistics · doi:10.1214/aos/1074290335 · Cited by 2,204
Bullard, J. H., Purdom, E., Hansen, K. D., & Dudoit, S. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-94 · Cited by 1,680
Dudoit, S., Yang, J. Y. H., Callow, M. J., & Speed, T. P. (2002) · Cited by 1,392
Analysis of Variance for Gene Expression Microarray Data
Kerr, K. F., Martin, M., & Churchill, G. A. (2000) · Journal of Computational Biology · doi:10.1089/10665270050514954 · Cited by 1,310
Small-sample estimation of negative binomial dispersion, with applications to SAGE data
Robinson, M. D., & Smyth, G. K. (2007) · Biostatistics · doi:10.1093/biostatistics/kxm030 · Cited by 1,073
[12] DNA arrays for analysis of gene expression
Eisen, M. B., & Brown, P. O. (1999) · Methods in enzymology on CD-ROM/Methods in enzymology · doi:10.1016/s0076-6879(99)03014-1 · Cited by 1,043
Moderated statistical tests for assessing differences in tag abundance
Robinson, M. D., & Smyth, G. K. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm453 · Cited by 856
baySeq: Empirical Bayesian methods for identifying differential expression in sequence count data
Hardcastle, T. J., & Kelly, K. A. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-422 · Cited by 755
't Hoen, P. A. C., Ariyurek, Y., Thygesen, H. H., Vreugdenhil, E., Vossen, R. H. A. M., de Menezes, R. X., Boer, J. M., van Ommen, G.-J. B., & den Dunnen, J. T. (2008) · Nucleic Acids Research · doi:10.1093/nar/gkn705 · Cited by 742
Newton, M. A., Kendziorski, C. M., Richmond, C. S., Blattner, F. R., & Tsui, K. (2001) · Journal of Computational Biology · doi:10.1089/106652701300099074 · Cited by 739
Biases in Illumina transcriptome sequencing caused by random hexamer priming
Hansen, K. D., Brenner, S. E., & Dudoit, S. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq224 · Cited by 713
The Analysis of Gene Expression Data: Methods and Software
Boyle, T. (2005) · Journal of Statistical Software · doi:10.18637/jss.v014.b01 · Cited by 513
RNA-Seq—quantitative measurement of expression through massively parallel RNA-sequencing
Wilhelm, B. T., & Landry, J. (2009) · Methods · doi:10.1016/j.ymeth.2009.03.016 · Cited by 500
Transcript length bias in RNA-seq data confounds systems biology
Oshlack, A., & Wakefield, M. J. (2009) · Biology Direct · doi:10.1186/1745-6150-4-14 · Cited by 497
The beginning of the end for microarrays?
Shendure, J. (2008) · Nature Methods · doi:10.1038/nmeth0708-585 · Cited by 331
Witten, D., Tibshirani, R. J., Gu, S. G., Fire, A. Z., & Lui, W. (2010) · BMC Biology · doi:10.1186/1741-7007-8-58 · Cited by 234
Self-citationModeling non-uniformity in short-read rates in RNA-Seq data
Li, J., Jiang, H., & Wong, W. H. (2010) · Genome Biology · doi:10.1186/gb-2010-11-5-r50 · Cited by 193
Self-citationBloom, J. S., Khan, Z., Kruglyak, L., Singh, M., & Caudy, A. A. (2009) · BMC Genomics · doi:10.1186/1471-2164-10-221 · Cited by 178
A two-parameter generalized Poisson model to improve the analysis of RNA-seq data
Srivastava, S., & Chen, L. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq670 · Cited by 161
Storey, J. D., & Tibshirani, R. J. (2003) · Statistics for biology and health · doi:10.1007/0-387-21679-0_12 · Cited by 148
Self-citationTrumbore, S. E., & Gaudinski, J. B. (2003) · Science · doi:10.1126/science.1091841 · Cited by 142
Lu, J., Tomfohr, J. K., & Kepler, T. B. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-165 · Cited by 79
Overdispersed logistic regression for SAGE: Modelling multiple groups and covariates
Baggerly, K., Deng, L., Morris, J. S., & Aldaz, C. M. (2004) · BMC Bioinformatics · doi:10.1186/1471-2105-5-144 · Cited by 58
Research works printed but not linked: 3
- 1995, JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B, 85, 289
- 2002, STATISTICA SINICA, 12, 111
- 2008, Science, 302, 1344
Cited by
Sorted by citations
featureCounts: an efficient general purpose program for assigning sequence reads to genomic features
Liao, Y., Smyth, G. K., & Shi, W. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btt656 · Cited by 27,219
voom: precision weights unlock linear model analysis tools for RNA-seq read counts
Law, C. W., Chen, Y., Shi, W., & Smyth, G. K. (2014) · Genome Biology · doi:10.1186/gb-2014-15-2-r29 · Cited by 6,155
“This article compares the performance of the limma-based pipelines to edgeR [20,34], DESeq [13], baySeq [21], TSPM [25], PoissonSeq [26] and DSS [22], all of which are based on NB or over-dispersed Poisson distributions.”
MethodInfluentialA survey of best practices for RNA-seq data analysis
Conesa, A., Madrigal, P., Tarazona, S., Gomez-Cabrero, D., Cervera, A., McPherson, A., Szcześniak, M. W., Gaffney, D. J., Elo, L. L., Zhang, X., et al. (2016) · Genome Biology · doi:10.1186/s13059-016-0881-8 · Cited by 2,794
“Normalization methods that take this into account are TMM [47], DESeq [48], PoissonSeq [49] and UpperQuartile [45], which ignore highly variable and/or highly expressed features.”
MethodSingle-cell RNA sequencing technologies and bioinformatics pipelines
Hwang, B., Lee, J. H., & Bang, D. (2018) · Experimental & Molecular Medicine · doi:10.1038/s12276-018-0071-8 · Cited by 1,866
“To overcome the inherent problems in within-sample normalization methods, alternative approaches have been developed 52 – 54 .”
MethodDifferential analysis of RNA-seq incorporating quantification uncertainty
Pimentel, H., Bray, N. L., Puente, S., Melsted, P., & Pachter, L. (2017) · Nature Methods · doi:10.1038/nmeth.4324 · Cited by 1,534
Metadata
- Authors
- Li, J., Witten, D. M., Johnstone, I. M., & Tibshirani, R.
- Affiliations
- Stanford University (US)
- University of Washington (US)
- Corresponding author
- Li, J.
- Venue
- Biostatistics
- Publisher
- Oxford University Press (OUP)
- Published
- 2011-10-14
- Volume
- 13
- Issue
- 3
- Pages
- 523-538
- ISSN
- 1465-4644 (ISSN-L)
- 1468-4357
- PMC
- PMC3372940
- OpenAlex
- W2142165994
- Cited by
- 369
- References
- 40
- Citations by year
- 2026: 3 · 2025: 10 · 2024: 12 · 2023: 22 · 2022: 22 · 2021: 49 · 2020: 30 · 2019: 24 · 2018: 24 · 2017: 33 · 2016: 29 · 2015: 39 · 2014: 39 · 2013: 19 · 2012: 12
- FWCI
- 9.36
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- Biostatistics (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (n01hv28183-13-0-0)
- National Heart, Lung, and Blood Institute (N01-HV-28 183)
- National Institute of Biomedical Imaging and Bioengineering (R01EB1988, R01EB001988)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Keywords
- RNA sequencing, read count normalization, false discovery rate estimation, log-linear models, significance testing, count data, Poisson model, negative binomial regression
- MeSH terms
- Data Interpretation, Statistical, Humans, RNA, Messenger, Models, Statistical, Sequence Analysis, DNA, Reverse Transcriptase Polymerase Chain Reaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
