The DOI matches its Crossref record; 22 of the 23 references it cites were found in a registry or an index.
Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments
Open accessAbstract
BACKGROUND: High-throughput sequencing technologies, such as the Illumina Genome Analyzer, are powerful new tools for investigating a wide range of biological and medical questions. Statistical and computational methods are key for drawing meaningful and accurate conclusions from the massive and complex datasets generated by the sequencers. We provide a detailed evaluation of statistical methods for normalization and differential expression (DE) analysis of Illumina transcriptome sequencing (mRNA-Seq) data. RESULTS: We compare statistical methods for detecting genes that are significantly DE between two types of biological samples and find that there are substantial differences in how the test statistics handle low-count genes. We evaluate how DE results are affected by features of the sequencing platform, such as, varying gene lengths, base-calling calibration method (with and without phi X control lane), and flow-cell/library preparation effects. We investigate the impact of the read count normalization method on DE results and show that the standard approach of scaling by total lane counts (e.g., RPKM) can bias estimates of DE. We propose more general quantile-based normalization procedures and demonstrate an improvement in DE detection. CONCLUSIONS: Our results have significant practical and methodological implications for the design and analysis of mRNA-Seq experiments. They highlight the importance of appropriate statistical methods for normalization and DE inference, to account for features of the sequencing platform that could impact the accuracy of results. They also reveal the need for further research in the development of statistical and computational methods for mRNA-Seq.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 4 Comparison of biological, library preparation, and flow-cell effects. Boxplots of estimated log-fold-changes for UHR vs. Brain biological effects (GLM 2 in [Additional file 2: Supplemental Table S4]), flow-cell effects adjusting for biology (GLM 4), library preparation effects within flow-cell (GLM 7). Estimates are presented for total-count (black) and upper-quartile (blue) normalization.](/paper-figures/7a218023bc1142f6aea616fd271add05/evaluation-of-statistical-methods-for-normalization-and-differential-expression-figure-4.jpg)
![Figure 6 Comparison of mRNA-Seq and microarray differential expression calls to qRT-PCR: ROC curves. Genes common to all three platforms and present for both qRT-PCR and sequencing (see [Additional file 2: Supplemental Section S6]). evaluated and declared DE if their qRT-PCR absolute log-ratio was (a) greater than 2 or (b) greater than 0.5; genes were declared non-DE if their absolute log-ratio was less than 0.2. The GLM-based likelihood ratio test was used for the sequencing data. Two normalization procedures are presented for mRNA-Seq: total-count (black) and upper-quartile (blue) normalization. Microarray data were normalized using RMA (gray). Note that we require a true positive to be differentially expressed in the same direction according to both mRNA-Seq and qRT-PCR (see Table 1 and Methods).](/paper-figures/e79890d44a604905bdf241f0104817db/evaluation-of-statistical-methods-for-normalization-and-differential-expression-figure-6.jpg)
![Figure 7 Comparison of mRNA-Seq and microarray differential expression measures to qRT-PCR. Difference scatterplots comparing the estimates of UHR/Brain expression log-ratio from qRT-PCR to those from (a) mRNA-Seq, using the standard total-count normalization, and (b) microarrays, using the standard RMA normalization. Shown are the genes shared between all three platforms, present in both Brain and UHR according to both mRNA-Seq and qRT-PCR (see [Additional file 2: Supplemental Section S1]), and having absolute qRT-PCR expression log-ratio less than 4. Horizontal lines in (a) represent the median UHR/Brain log-ratio for the sequencing data after the standard total-count normalization (black), POLR2A normalization (red), quantile normalization (yellow), upper-quartile normalization (blue); horizontal lines in (b) show the median UHR/Brain log-ratio for the microarray data after the standard RMA normalization (black) and POLR2A normalization (red).](/paper-figures/ffe14db2d64a46939165f0c6e8ca7bec/evaluation-of-statistical-methods-for-normalization-and-differential-expression-figure-7.jpg)
![Figure 8 Comparison of normalization procedures: Goodness-of-fit of Poisson model. The multiplicative Poisson model (GLM 1 in [Additional file 2: Supplemental Table S4]) is fit to the seven Brain lanes in the MAQC-2 experiment after (a) total-count, (b) POLR2A, (c) upper-quartile, and (d) quantile normalization. Goodness-of-fit statistics are computed and displayed in χ2 quantile-quantile plots. Genes with goodness-of-fit statistics in the top quantiles of the χ2-distribution are displayed using colored plotting symbols: red (1, 5]%, purple (.1, 1]%, gold [0, .1]%. Similar plots for UHR show the same patterns.](/paper-figures/d99eb5ce90ee4a9ebac8c9f438b4c283/evaluation-of-statistical-methods-for-normalization-and-differential-expression-figure-8.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These approaches rely on normalizing methods that are based on total or effective counts, and tend to perform poorly when samples have heterogeneous transcript distributions, that is, when highly and differentially expressed features can skew the count distribution [45, 46].”
From A survey of best practices for RNA-seq data analysis (2016) · cites this paper as [45]
Experimental — accuracy not yet measuredIn the citing paper
“While reproducibility in RNA-seq is usually high at the level of sequencing [1, 45], other steps such as RNA extraction and library preparation are noisier and may introduce biases in the data that can be minimized by adopting good experimental procedures (Box 2).”
From A survey of best practices for RNA-seq data analysis (2016) · cites this paper as [45]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ultrafast and memory-efficient alignment of short DNA sequences to the human genome.
Langmead, B., Trapnell, C., Pop, M., & Salzberg, S. L. (2009) · Genome Biology · doi:10.1186/gb-2009-10-3-r25 · Cited by 21,849 · cited 1×
“We used Bowtie to map reads to the genome (GRCh37 assembly) [ 16 ].”
Method(2005) · Springer eBooks · doi:10.1007/0-387-27255-0_9 · Cited by 20,333
Mapping and quantifying mammalian transcriptomes by RNA-Seq
Mortazavi, A., Williams, B. A., McCue, K., Schaeffer, L., & Wold, B. (2008) · Nature Methods · doi:10.1038/nmeth.1226 · Cited by 13,696 · cited 6×
“These high-throughput sequencing technologies have already been applied to monitor genome-wide transcription levels (mRNA-Seq), DNA-protein interactions (ChIP-Seq), chromatin structure, and DNA methylation [1-9].”
BackgroundMethodInfluentialSmyth, G. K. (2004) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1027 · Cited by 11,829
Exploration, normalization, and summaries of high density oligonucleotide array probe level data
Irizarry, R. A. (2003) · Biostatistics · doi:10.1093/biostatistics/4.2.249 · Cited by 10,603
Base-Calling of Automated Sequencer Traces Using Phred. I. Accuracy AssessmentPreprint
Ewing, B., Hillier, L., Wendl, M. C., & Green, P. (1998) · Genome Research · doi:10.1101/gr.8.3.175 · Cited by 6,993
Base-Calling of Automated Sequencer Traces Using Phred. II. Error Probabilities
Ewing, B., & Green, P. (1998) · Genome Research · doi:10.1101/gr.8.3.186 · Cited by 5,439 · cited 1×
“quality scores for the other seven lanes [[15], Supplementary Information, p. 7]. Bustard can also be run using the auto-calibration method, which scores basecalls in a manner similar to the phred base-caller [ 17 ] and does not require a control lane per flow-cell.”
MethodAlternative isoform regulation in human tissue transcriptomes
Wang, E. T., Sandberg, R., Luo, S., Khrebtukova, I., Zhang, L., Mayr, C., Kingsmore, S. F., Schroth, G. P., & Burge, C. B. (2008) · Nature · doi:10.1038/nature07509 · Cited by 5,181
Accurate whole human genome sequencing using reversible terminator chemistry
Bentley, D. R., Balasubramanian, S., Swerdlow, H. P., Smith, G. P., Milton, J., Brown, C. G., Hall, K. P., Evers, D. J., Barnes, C. L., Bignell, H. R., et al. (2008) · Nature · doi:10.1038/nature07517 · Cited by 3,684 · cited 1×
“quality scores for the other seven lanes [[15], Supplementary Information, p.”
BackgroundRNA-seq: An assessment of technical reproducibility and comparison with gene expression arrays
Marioni, J. C., Mason, C. E., Mane, S. M., Stephens, M., & Gilad, Y. (2008) · Genome Research · doi:10.1101/gr.079558.108 · Cited by 2,771 · cited 7×
“These high-throughput sequencing technologies have already been applied to monitor genome-wide transcription levels (mRNA-Seq), DNA-protein interactions (ChIP-Seq), chromatin structure, and DNA methylation [1-9].”
BackgroundMethodResultInfluential
Research works printed but not linked: 4
- Illumina:Sequencing Analysis Software User Guide For Pipeline Version 1.3 and CASAVA Version 1.0 T. Illumina, Inc.; 2008. [Part # 1005359 Rev. A] [ http://icom.illumina.com/icom/software.ilmn?id=277 ] [Part # 1005359 Rev. A]
- Illumina:Preparing Samples for Sequencing mRNA. Ilumina, Inc.; 2009. [Part # 1004898 Rev. A] [ http://icom.illumina.com/icom/software.ilmn?id=277 ] [Part # 1004898 Rev. A]
- Taub MA: Analysis of high-throughput biological data: some statistical problems in RNA-seq and mouse genotyping. PhD thesis. Department of Statistics, UC Berkeley; 2009.
- Irizarry RA, Hobbs B, Collin F, Beazer-Barclay YD, Antonellis KJ, Scherf U, Speed TP: Exploration, normalization, and summaries of high density oligonucleotide array probe level data. Biostatistics 2003, 4(1465–4644 (Print)):249–64. 10.1093/biostatistics/4.2.249 · DOI · Paper page
Cited by
Sorted by citations
Items 501–600 of 999
Tardu, M., Bulut, S., & Kavaklı, İ. H. (2017) · Scientific Reports · doi:10.1038/srep40817 · Cited by 43
A comprehensive simulation study on classification of RNA-Seq data
Zararsız, G., Göksülük, D., Korkmaz, S., Eldem, V., Zararsız, G. E., Duru, İ. P., & Ozturk, A. B. (2017) · PLoS ONE · doi:10.1371/journal.pone.0182507 · Cited by 43
van Rooij, J., Mandaviya, P. R., Claringbould, A., Felix, J. F., van Dongen, J., Jansen, R., Franke, L., Hoen, P. A. C. ’., Heijmans, B. T., & van Meurs, J. B. J. (2019) · Genome biology · doi:10.1186/s13059-019-1878-x · Cited by 42
Data-based RNA-seq simulations by binomial thinning
Gerard, D. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-3450-9 · Cited by 43
Fanidis, D., & Moulos, P. (2020) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa156 · Cited by 41
Swindell, W. R., Xing, X., Voorhees, J. J., Elder, J. T., Johnston, A., & Gudjonsson, J. E. (2014) · Physiological Genomics · doi:10.1152/physiolgenomics.00022.2014 · Cited by 42
Different biological effects of exposure to far-UVC (222 nm) and near-UVC (254 nm) irradiation
Tavares, R. S. N., Adamóski, D., Girasole, A., Lima, E. N., Justo-Júnior, A. D. S., Domingues, R. R., Silveira, A. C. C., Marques, R. E., Carvalho, M. D. O., Ambrosio, A. L. B., et al. (2023) · Journal of Photochemistry and Photobiology B Biology · doi:10.1016/j.jphotobiol.2023.112713 · Cited by 47
Díez, C. M., Meca, E., Tenaillon, M. I., & Gaut, B. S. (2014) · PLoS Genetics · doi:10.1371/journal.pgen.1004298 · Cited by 41
Rutledge, E. A., Bénazet, J.-D., & McMahon, A. P. (2017) · Development · doi:10.1242/dev.149112 · Cited by 39
“Upper quartile normalization was used to normalize the read counts per gene/ transcript in all samples (Bullard et al., 2010).”
MethodMoorey, S. E., Griffin, B. A., Elmore, M. F., Kriese-Anderson, L. A., Elmore, J. B., Dyce, P. W., Rodning, S. P. P., & Biase, F. H. (2018) · BMC Genomics · doi:10.1186/s12864-018-4505-4 · Cited by 40
“In order to remove quantification uncertainty associated to lowly expressed genes and erroneous identification of differentially expressed genes [40, 41], we retained genes with more than one count per million (1 CPM) in six or more samples for downstream analyses, for each location independently.”
BackgroundJiang, S., Xiao, G., Koh, A. Y., Kim, J., Li, Q., & Zhan, X. (2019) · Biostatistics · doi:10.1093/biostatistics/kxz050 · Cited by 42
Negative binomial additive model for RNA-Seq data analysis
Ren, X., & Kuan, P. F. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-3506-x · Cited by 42
“Several methods have been developed for RNA-Seq data normalization including total count, upper quartile [17], median, quantile [18, 19], conditional quantile [20], reads per kilobase per million mapped reads (RPKM) [21], median-of-ratios [11], and trimmed mean of M values [22].”
MethodKadota, K., & Shimizu, K. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-227 · Cited by 43
Optimizing a Massive Parallel Sequencing Workflow for Quantitative miRNA Expression Analysis
Cordero, F., Beccuti, M., Arigoni, M., Donatelli, S., & Calogero, R. (2012) · PLoS ONE · doi:10.1371/journal.pone.0031630 · Cited by 41
Statistical Analyses of Next Generation Sequence Data: A Partial Overview
Datta, S., Datta, S., Kim, S., Chakraborty, S., & Gill, R. S. (2010) · Journal of Proteomics & Bioinformatics · doi:10.4172/jpb.1000138 · Cited by 41
Spatially Distinct Neutrophil Responses within the Inflammatory Lesions of Pneumonic Plague
Stasulli, N. M., Eichelberger, K. R., Price, P. A., Pechous, R. D., Montgomery, S. A., Parker, J. S., & Goldman, W. E. (2015) · mBio · doi:10.1128/mbio.01530-15 · Cited by 42
Avian transcriptomics: opportunities and challenges
Jax, E., Wink, M., & Kraus, R. H. (2018) · Journal für Ornithologie · doi:10.1007/s10336-018-1532-5 · Cited by 44
“This can be done using scale normalisation factors such as the trimmed mean of M values (Robinson and Oshlack 2010), the upper quartile (Bullard et al. 2010), or the geometric mean (Anders and Huber 2010).”
BackgroundMethodInfluentialHan, W., Yang, F., Wu, Z., Guo, F., Zhang, J., Hai, E., Shang, F., Su, R., Wang, R., Wang, Z., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-60351-5 · Cited by 42
Li, B., & Dewey, C. N. (2014) · Apple Academic Press eBooks · doi:10.1201/b16589-5 · Cited by 41
Aberrant PRDM9 expression impacts the pan-cancer genomic landscapePreprint
Houle, A. A., Gibling, H., Lamaze, F. C., Edgington, H. A., Soave, D. M., Favé, M.-J., Agbessi, M. D. T., Bruat, V., Stein, L. D., & Awadalla, P. (2018) · Genome Research · doi:10.1101/gr.231696.117 · Cited by 41
“S2B), we found that the twometrics were highly correlated (Pearson’s correlation coefficient = 0.98, P 2.2 ×10−16), although 246 samples expressed PRDM9 above a threshold of 10 FPKM-UQ only in the uncorrected samples.”
MethodInfluentialMicroRNA expression profiles across blood and different tissues in cattle
Sun, H., Chen, Y., & Guan, L. L. (2019) · Scientific Data · doi:10.1038/sdata.2019.13 · Cited by 41
Common applications of next-generation sequencing technologies in genomic research
Lee, C., Chiu, Y., Wang, L.-B., Kuo, Y.-L., Chuang, E. Y.-Y., Lai, L., & Tsai, M. (2013) · Translational Cancer Research · doi:10.21037/962 · Cited by 40
Ruble, C., Smith, R. M., Calley, J. N., Munsie, L. M., Airey, D. C., Gao, Y., Shin, J. H., Hyde, T. M., Straub, R. E., Weinberger, D. R., et al. (2016) · BMC Genetics · doi:10.1186/s12863-015-0325-6 · Cited by 41
“However, it has been shown that Poisson distribution does not capture biological variation in read counts [69–71].”
BackgroundDosage compensation in the African malaria mosquitoAnopheles gambiae
Rose, G., Krzywińska, E., Kim, J., Revuelta, L., Luca Ferretti, A., & Krzywinski, J. (2016) · Genome Biology and Evolution · doi:10.1093/gbe/evw004 · Cited by 40
“As an alternative to total count normalization, we employed a quantile normalization (75th percentile normalization suggested in Bullard et al. 2010) to reduce the effects of outliers on the RNA-seq data.”
BackgroundMethodGene expression variability and the analysis of large-scale RNA-seq studies with the MDSeq
Ran, D., & Daye, Z. J. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkx456 · Cited by 40
“The MDSeq also allows the user to apply other normalization factors, including the relative log expression (RLE) (29), upper-quartile (46), and conditional quantile normalization (cqn) (82) factors.”
BackgroundMethodZhang, X., Li, L., Jiang, H., Jing, E., Li, J., & Chen, J. (2018) · Gene · doi:10.1016/j.gene.2018.03.014 · Cited by 41
The EXPANDER Integrated Platform for Transcriptome Analysis
Hait, T. A., Maron‐Katz, A., Sagir, D., Amar, D., Ulitsky, I., Linhart, C., Tanay, A., Sharan, R., Shiloh, Y., Elkon, R., et al. (2019) · Journal of Molecular Biology · doi:10.1016/j.jmb.2019.05.013 · Cited by 39
S-MART, A Software Toolbox to Aid RNA-seq Data Analysis
Zytnicki, M., & Quesneville, H. (2011) · PLoS ONE · doi:10.1371/journal.pone.0025988 · Cited by 39
ISOFORM ABUNDANCE INFERENCE PROVIDES A MORE ACCURATE ESTIMATION OF GENE EXPRESSION LEVELS IN RNA-SEQ
Wang, X., Wu, Z., & Zhang, X. (2010) · Journal of Bioinformatics and Computational Biology · doi:10.1142/s0219720010005178 · Cited by 39
Characterization and Comparison of the Leukocyte Transcriptomes of Three Cattle Breeds
Huang, W., Nadeem, A., Zhang, B., Babar, M. E., Soller, M., & Khatib, H. (2012) · PLoS ONE · doi:10.1371/journal.pone.0030244 · Cited by 39
Hu, Y., Liu, Y., Mao, X., Cheng, J., Ferguson, J. F., Xue, C., Reilly, M. P., Li, H., & Li, M. (2013) · Nucleic Acids Research · doi:10.1093/nar/gkt1304 · Cited by 38
Differential expression--the next generation and beyond
Auer, P. L., Srivastava, S., & Doerge, R. W. (2011) · Briefings in Functional Genomics · doi:10.1093/bfgp/elr041 · Cited by 39
RNA‐Seq Highlights High Clonal Variation in Monoclonal Antibody Producing CHO Cells
Orellana, C. A., Marcellin, E., Palfreyman, R. W., Munro, T. P., Gray, P. P., & Nielsen, L. K. (2018) · Biotechnology Journal · doi:10.1002/biot.201700231 · Cited by 41
Robust identification of differentially expressed genes from RNA-seq data
Shahjaman, M., Mollah, M. M. H., Rahman, M. R., Islam, S. M. S., & Mollah, M. N. H. (2019) · Genomics · doi:10.1016/j.ygeno.2019.11.012 · Cited by 41
Yang, C., & Wei, H. (2014) · Molecular Plant · doi:10.1016/j.molp.2014.11.012 · Cited by 41
Romero–Campero, F. J., Pérez–Hurtado, I., Lucas-Reina, E., Romero, J., & Valverde, F. (2016) · BMC Genomics · doi:10.1186/s12864-016-2564-y · Cited by 38
“The selection of differentially expressed genes was performed using the standard methodology applied to the analysis of RNA-seq data described in [97].”
MethodLeishman, E., Murphy, M., Mackie, K., & Bradshaw, H. B. (2018) · Biochimica et Biophysica Acta (BBA) - Molecular and Cell Biology of Lipids · doi:10.1016/j.bbalip.2018.02.001 · Cited by 42
“The extracted RNA was purified into poly-A RNA using oligo-dT binding [50] and fragmented into 30-400 base pair segments [51].”
MethodHER2-mTOR signaling–driven breast cancer cells require ER-associated degradation to survive
Singh, N., Joshi, R., & Komurov, K. (2015) · Science Signaling · doi:10.1126/scisignal.aaa6922 · Cited by 39
“The normalization method of RNAseq data adopted by the TCGA follows an upper quartile normalization method, which, unlike the more popular RPKM method, does not assume equal global mRNA content between samples and therefore enables genome-wide comparison of abundances of highly expressed mRNAs between samples (40).”
MethodLiu, T., Fang, Z., Zhang, Z., Yu, Y., Li, M., & Yin, M. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.11.055 · Cited by 42
“Enriched SVGs or meta-genes are identified through the analysis of differential expressions (DEs) [45].”
BackgroundRobinson, D. G., Wang, J. Y., & Storey, J. D. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv636 · Cited by 38
Metabolic studies in plant organs: don't forget dilution by growth
Génard, M., Baldazzi, V., & Gibon, Y. (2014) · Frontiers in Plant Science · doi:10.3389/fpls.2014.00085 · Cited by 39
Accuracy of RNA-Seq and its dependence on sequencing depth
Cai, G., Li, H., Lu, Y., Huang, X., Lee, J. H., Müller, P., Ji, Y., & Liang, S. (2012) · BMC Bioinformatics · doi:10.1186/1471-2105-13-s13-s5 · Cited by 39
Analysis of the transcriptome in molecular epidemiology studies
McHale, C. M., Zhang, L., Thomas, R., & Smith, M. T. (2013) · Environmental and Molecular Mutagenesis · doi:10.1002/em.21798 · Cited by 38
Tsementzi, D., Poretsky, R. S., Rodriguez‐R, L. M., Luo, C., & Konstantinidis, K. T. (2014) · Environmental Microbiology Reports · doi:10.1111/1758-2229.12180 · Cited by 40
Tamboli, R. A., Hajri, T., Jiang, A., Marks–Shulman, P. A., Williams, D. B., Clements, R. H., Melvin, W. V., Bowen, B. P., Shyr, Y., Abumrad, N. N., et al. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028577 · Cited by 38
Transcriptomic profiles of high and low antibody responders to smallpox vaccine
Kennedy, R. B., Oberg, A. L., Ovsyannikova, I. G., Haralambieva, I. H., Grill, D. E., & Poland, G. A. (2013) · Genes and Immunity · doi:10.1038/gene.2013.14 · Cited by 37
Marcellin, E., Mercer, T. R., Licona‐Cassani, C., Palfreyman, R. W., Dinger, M. E., Steen, J. A., Mattick, J. S., & Nielsen, L. K. (2013) · BMC Genomics · doi:10.1186/1471-2164-14-15 · Cited by 37
Zhang, J., Jiang, D., Liu, B., Luo, W., Lü, J., Ma, T., & Wan, D. (2014) · Plant Cell Reports · doi:10.1007/s00299-014-1638-z · Cited by 37
McKinney, B. A., White, B. C., Grill, D. E., Li, P. W., Kennedy, R. B., Poland, G. A., & Oberg, A. L. (2013) · PLoS ONE · doi:10.1371/journal.pone.0081527 · Cited by 37
Beckers, M., Mohorianu, I. I., Stocks, M. B., Applegate, C., Dalmay, T., & Moulton, V. (2017) · RNA · doi:10.1261/rna.059360.116 · Cited by 36
“…per total (RPT) method (Mortazavi et al. 2008) for which the total abundance of all reads in a sample is considered, upper quartile normalization (Bullard et al. 2010) for which only the reads with abundances in the upper quartile are considered, the trimmed mean of M-values (TMM) (Anders and…”
Ferté, C., Trister, A. D., Huang, E. S., Bot, B. M., Guinney, J., Commo, F., Sieberts, S. K., André, F., Besse, B., Soria, J., et al. (2013) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-12-3937 · Cited by 36
Targeted RNA-Sequencing with Competitive Multiplex-PCR Amplicon Libraries
Blomquist, T. M., Crawford, E. L., Lovett, J. L., Yeo, J., Stanoszek, L. M., Levin, A. M., Li, J., Lu, M., Shi, L. M., Muldrew, K. L., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0079120 · Cited by 37
Li, X., Kanakala, S., He, Y., Zhong, X., Yu, S., Li, R., Sun, L., & Ma, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0169838 · Cited by 37
Global insights into the Chinese hamster and CHO cell transcriptomes
Vishwanathan, N., Yongky, A., Johnson, K. C., Fu, H., Jacob, N. M., Le, H., Yusufi, F. N. K., Lee, D., & Hu, W. (2014) · Biotechnology and Bioengineering · doi:10.1002/bit.25513 · Cited by 39
“The number of readsmapped to a contig was normalized across all eight libraries to adjust for the variability in sequencing depth using the upper quantile normalization procedure (Bullard et al., 2010).”
MethodBrief guide to RNA sequencing analysis for nonexperts in bioinformatics
Lee, G., Ham, S., & Lee, S. (2024) · Molecules and Cells · doi:10.1016/j.mocell.2024.100060 · Cited by 35
Li, S., Dong, X., & Su, Z. (2013) · BMC Genomics · doi:10.1186/1471-2164-14-520 · Cited by 40
Evaluating statistical analysis models for RNA sequencing experiments
Reeb, P., & Steibel, J. P. (2013) · Frontiers in Genetics · doi:10.3389/fgene.2013.00178 · Cited by 36
Technical and biological variance structure in mRNA-Seq data: life in the real world
Oberg, A. L., Bot, B. M., Grill, D. E., Poland, G. A., & Therneau, T. M. (2012) · BMC Genomics · doi:10.1186/1471-2164-13-304 · Cited by 36
Single-Cell Transcriptome Study as Big Data
Yu, P., & Lin, W. (2016) · Genomics Proteomics & Bioinformatics · doi:10.1016/j.gpb.2016.01.005 · Cited by 36
“Reads per million mapped reads (RPM), reads per kilobase per million mapped reads (RPKM) [92], median, and upper-quantile normalizations [93] rescale the raw counts based on mean, mean with gene length considered, median, and upper-quantile of read counts, respectively, in a sample.”
BackgroundWang, Y., Xu, B., Ning, S., Shi, S., & Tan, L. (2020) · Ecotoxicology and Environmental Safety · doi:10.1016/j.ecoenv.2020.111791 · Cited by 36
Sav1 Loss Induces Senescence and Stat3 Activation Coinciding with Tubulointerstitial Fibrosis
Leung, J. Y., Wilson, H. L., Voltzke, K. J., Williams, L. A., Lee, H. J., Wobker, S. E., & Kim, W. Y. (2017) · Molecular and Cellular Biology · doi:10.1128/mcb.00565-16 · Cited by 37
“Raw RSEM read counts or all RNA-seq samples were normalized to the overall upper quartile (69).”
MethodChen, B., Chen, X., Wu, X., Wang, X., Wang, Y., Lin, T., Kurata, J. S., Wu, J., Vonderfecht, S. L., Sun, G., et al. (2014) · Cancer Letters · doi:10.1016/j.canlet.2014.09.034 · Cited by 36
Hardcastle, T. J., & Kelly, K. A. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-135 · Cited by 35
Using Poisson mixed-effects model to quantify transcript-level gene expression in RNA-Seq
Hu, M., Zhu, Y., Taylor, J. M. G., Liu, J. S., & Qin, Z. S. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr616 · Cited by 35
Segmentor3IsBack: an R package for the fast and exact segmentation of Seq-data
Cleynen, A., Koskas, M., Lebarbier, É., Rigaill, G. J., & Robin, S. S. (2014) · Algorithms for Molecular Biology · doi:10.1186/1748-7188-9-6 · Cited by 41
RNA-seq analysis in forest tree species: bioinformatic problems and solutions
de Heredia, U. L., & Vázquez‐Poletti, J. L. (2016) · Tree Genetics & Genomes · doi:10.1007/s11295-016-0995-x · Cited by 40
“The upper quantile (UQ) normalization (Bullard et al. 2010) replaces the total counts by the upper quantile of the counts.”
BackgroundSePIA: RNA and small RNA sequence processing, integration, and analysis
Icay, K., Chen, P., Cervera, A., Rantanen, V., Lehtonen, R., & Hautaniemi, S. (2016) · BioData Mining · doi:10.1186/s13040-016-0099-z · Cited by 30
“While the former two represent standard scaling approaches, the latter is a robust alternative for more complex datasets [16].”
BackgroundImproving cross-study prediction through addon batch effect adjustment or addon normalization
Hornung, R., Causeur, D., Bernau, C., & Boulesteix, A. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw650 · Cited by 35
Computational approaches for high‐throughput single‐cell data analysis
Todorov, H., & Saeys, Y. (2018) · FEBS Journal · doi:10.1111/febs.14613 · Cited by 35
Yang, X., Cheng, H. W., Chen, J., Wang, R., Saleh, A. D., Si, H., Lee, S. S., Guven‐Maiorov, E., Keskin, Ö., Gürsoy, A., et al. (2019) · Cancer Immunology Research · doi:10.1158/2326-6066.cir-18-0832 · Cited by 36
“0) and upper quartile normalization method (28).”
HARMONIES: A Hybrid Approach for Microbiome Networks Inference via Exploiting Sparsity
Jiang, S., Xiao, G., Koh, A. Y., Chen, Y., Yao, B., Li, Q., & Zhan, X. (2020) · Frontiers in Genetics · doi:10.3389/fgene.2020.00445 · Cited by 36
“Similarly, other methods have been proposed with different constraints for normalizing the sequencing data Robinson and Oshlack (2010); Anders and Huber (2010); Bullard et al. (2010); Paulson et al. (2013).”
MethodA comprehensive survey on computational learning methods for analysis of gene expression data
Bhandari, N., Walambe, R. A., Kotecha, K. V., & Khare, S. (2022) · Frontiers in Molecular Biosciences · doi:10.3389/fmolb.2022.907150 · Cited by 40
“Loess is a local polynomial regression-based approach which can be utilized to adjust intensity levels between two channels (Yang et al., 2002; Smyth and Speed, 2003; Bullard et al., 2010; Baans et al., 2017).”
MethodA User’s Guide to Quantitative and Comparative Analysis of Metagenomic Datasets
Luo, C., Rodriguez‐R, L. M., & Konstantinidis, K. T. (2013) · Methods in enzymology on CD-ROM/Methods in enzymology · doi:10.1016/b978-0-12-407863-5.00023-x · Cited by 35
RNA-SEQ: A GLANCE AT TECHNOLOGIES AND METHODOLOGIES
Jazayeri, S. M., Melgarejo, L. M., & Romero, H. M. (2014) · Acta Biológica Colombiana · doi:10.15446/abc.v20n2.43639 · Cited by 35
Network-Based Identification of Biomarkers Coexpressed with Multiple Pathways
Guo, N. L., & Wan, Y. (2014) · Cancer Informatics · doi:10.4137/cin.s14054 · Cited by 35
The MEF2A transcription factor interactome in cardiomyocytes
Moustafa, A., Hashemi, S., Brar, G. S., Grigull, J., Ng, S. H. S., Williams, D., Schmitt‐Ulms, G., & McDermott, J. C. (2023) · Cell Death and Disease · doi:10.1038/s41419-023-05665-8 · Cited by 34
Optimal Scaling of Digital Transcriptomes
Glusman, G., Caballero-Pérez, J., Robinson, M., Kutlu, B., & Hood, L. (2013) · PLoS ONE · doi:10.1371/journal.pone.0077885 · Cited by 33
Rathe, S. K., Moriarity, B. S., Stoltenberg, C. B., Kurata, M., Aumann, N. K., Rahrmann, E. P., Bailey, N. J., Melrose, E. G., Beckmann, D. A., Liska, C. R., et al. (2014) · Scientific Reports · doi:10.1038/srep06048 · Cited by 34
Genome and Transcriptome Analysis of the Food-Yeast Candida utilis
Tomita, Y., Ikeo, K., Tamakawa, H., Gojobori, T., & Ikushima, S. (2012) · PLoS ONE · doi:10.1371/journal.pone.0037226 · Cited by 35
Torres-Oliva, M., Almudí, I., McGregor, A. P., & Posnien, N. (2016) · BMC Genomics · doi:10.1186/s12864-016-2646-x · Cited by 35
“However, it has been shown that even after correcting for length differences, a longer transcript is more likely to appear as differentially expressed if RPKM values are used to assess the statistical significance [35, 37, 47 – 49].”
BackgroundMethodCao, Z., & Zhang, S. (2016) · Scientific Reports · doi:10.1038/srep33398 · Cited by 34
“The other is upper quartile (UQ) normalization 57 .”
InfluentialGE-mini: a mobile APP for large-scale gene expression visualization
Tang, Z., Li, C., Zhang, K., Yang, M., & Hu, X. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw775 · Cited by 34
Comparison of alternative approaches for analysing multi-level RNA-seq data
Mohorianu, I. I., Bretman, A. J., Smith, D. T., Fowler, E. K., Dalmay, T., & Chapman, T. (2017) · PLoS ONE · doi:10.1371/journal.pone.0182694 · Cited by 32
Matt, G. Y., & Umen, J. (2017) · G3 Genes Genomes Genetics · doi:10.1534/g3.117.300253 · Cited by 33
“Expression ratios derived from DESeq2 and RPKM normalization were highly correlated ( r 2 = 0.995) (Figure S9 in File S1), and RPKM expression values were used for all further analyses and reported in Datasets S1, S2, S3, S4, S5, S6, S7, S8, S9 and S10.”
InfluentialPoulard, C., Baulu, E., Lee, B. H., Pufall, M. A., & Stallcup, M. R. (2018) · Cell Death and Disease · doi:10.1038/s41419-018-1110-z · Cited by 34
Kappeler, B., Regitano, L. C. D. A., Poleti, M., César, A. S. M., Moreira, G. C. M., Gasparin, G. R., & Coutinho, L. L. (2019) · BMC Molecular Biology · doi:10.1186/s12867-018-0118-3 · Cited by 34
Wang, X., Wang, Y., Ning, S., Shi, S., & Tan, L. (2020) · Frontiers in Microbiology · doi:10.3389/fmicb.2020.00712 · Cited by 34
Dam, B., Dam, S., Kim, Y., & Liesack, W. (2013) · Environmental Microbiology · doi:10.1111/1462-2920.12367 · Cited by 34
Temperature threshold of isoprene emission from tropical trees, Ficus virgata and Ficus septica
Oku, H., Inafuku, M., Takamine, T., NAGAMINE, M., Saitoh, S., & Fukuta, M. (2013) · Chemosphere · doi:10.1016/j.chemosphere.2013.09.003 · Cited by 34
Renaut, S., Grassa, C. J., Moyers, B. T., Kane, N. C., & Rieseberg, L. H. (2012) · Biology · doi:10.3390/biology1030575 · Cited by 34
Newman, J. R. B., Conesa, A., Mika, M., New, F. N., Önengüt-Gümüşcü, S., Atkinson, M. A., Rich, S. S., McIntyre, L. M., & CONCANNON, P. J. (2017) · Genome Research · doi:10.1101/gr.217984.116 · Cited by 33
SIBER: systematic identification of bimodally expressed genes using RNAseq data
Tong, P., Chen, Y., Su, X., & Coombes, K. R. (2013) · Bioinformatics · doi:10.1093/bioinformatics/bts713 · Cited by 33
Radhakrishnan, S., Literman, R., Neuwald, J. L., & Valenzuela, N. (2018) · Sexual Development · doi:10.1159/000492188 · Cited by 34
“In Fisher’s exact test n ki is the observed read count for focal gene X (k = 1) or for all other genes in the transcriptome (k = 2) for treatment i ( i = 1 for 26 ° C and i = 2 for 31 ° C); n 11 + n 12 is the marginal total for focal gene X; n 21 + n 22 is the marginal total for the remaining genes in the transcriptom…”
MethodInfluentialLiu, W., Yu, Y., Shi-Yuan, C., Niu, X.-N., Jiang, W., Liu, G., Jiang, B., Tang, D., Lu, G., He, Y.-Q., et al. (2013) · Research in Microbiology · doi:10.1016/j.resmic.2013.02.005 · Cited by 33
Høvik, H., Yu, W., Olsen, I., & Chen, T. (2011) · Journal of Bacteriology · doi:10.1128/jb.06385-11 · Cited by 33
Durán, R. C., Wei, H., & Wu, J. (2024) · BMC Genomics · doi:10.1186/s12864-024-10364-5 · Cited by 37
“These methods will generate an over-correction for genes with low to moderate expression as well as an under-normalization for highly expressed genes [66].”
BackgroundMethodGatto, A., Torroja, C., Mazzarotto, F., Cook, S. A., Barton, P. J. R., Sánchez‐Cabo, F., & Lara‐Pezzi, E. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku166 · Cited by 34
Linville, J. L., Rodríguez‐Barranco, M., Land, M., Syed, M. H., Engle, N. L., Tschaplinski, T. J., Mielenz, J. R., & Cox, C. D. (2013) · PLoS ONE · doi:10.1371/journal.pone.0078829 · Cited by 32
McClure, R., Tjaden, B. C., & Genco, C. A. (2014) · Frontiers in Microbiology · doi:10.3389/fmicb.2014.00456 · Cited by 32
A survey of statistical software for analysing RNA-seq data
Gao, D., Kim, J., Kim, H., Phang, T. L., Selby, H. M., Tan, A. C., & Tong, T. (2010) · Human Genomics · doi:10.1186/1479-7364-5-1-56 · Cited by 32
Metadata
- Authors
- Bullard, J. H., Purdom, E., Hansen, K. D., & Dudoit, S.
- Affiliations
- University of California, Berkeley (US)
- Corresponding author
- Bullard, James H
- Venue
- BMC Bioinformatics
- Published
- 2010-02-18
- Volume
- 11
- Issue
- 1
- Pages
- 94-94
- ISSN
- 1471-2105 (ISSN-L)
- PMC
- PMC2838869
- OpenAlex
- W2156631105
- Cited by
- 1,680
- References
- 23
- Citations by year
- 2026: 29 · 2025: 51 · 2024: 69 · 2023: 68 · 2022: 92 · 2021: 132 · 2020: 127 · 2019: 123 · 2018: 123 · 2017: 133 · 2016: 121 · 2015: 129 · 2014: 161 · 2013: 128 · 2012: 98
- FWCI
- 41.79
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Bioinformatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Collection of Biostatistics Research Archive (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- National Science Foundation
- National Institutes of Health
- National Human Genome Research Institute (U01HG004271)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Keywords
- mRNA-seq, read count normalization, differential gene expression, method comparison
- MeSH terms
- RNA, Messenger, Sequence Analysis, RNA, Computational Biology, Databases, Genetic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
