Cited by 40,458Gene expression and cancer classificationGenomics and Phylogenetic StudiesBioinformatics and Genomic Networks

limma powers differential expression analyses for RNA-sequencing and microarray studies

Open access
Ritchie, Matthew E., Phipson, Belinda, Wu, Di, Hu, Yifang, Law, Charity W., Shi, Wei, Smyth, Gordon K.·2015·Nucleic Acids Research

Abstract

limma is an R/Bioconductor software package that provides an integrated solution for analysing data from gene expression experiments. It contains rich features for handling complex experimental designs and for information borrowing to overcome the problem of small sample sizes. Over the past decade, limma has been a popular choice for gene discovery through differential expression analyses of microarray and high-throughput PCR data. The package contains particularly strong facilities for reading, normalizing and exploring such data. Recently, the capabilities of limma have been significantly expanded in two important directions. First, the package can now perform both differential expression and differential splicing analyses of RNA sequencing (RNA-seq) data. All the downstream analysis tools previously restricted to microarray data are now available for RNA-seq as well. These capabilities allow users to analyse both RNA-seq and microarray data with very similar pipelines. Second, the package is now able to go past the traditional gene-wise expression analyses in a variety of ways, analysing expression profiles in terms of co-regulated sets of genes or in terms of higher-order expression signatures. This provides enhanced possibilities for biological interpretation of gene expression differences. This article reviews the philosophy and design of the limma package, summarizing both new and historical features, with an emphasis on recent enhancements and features that have not been previously described.

AI snapshot

Written from the paper’s full text.

TL;DR

limma is an R/Bioconductor package that provides an integrated linear-model and empirical Bayes framework for differential expression analysis across microarray, PCR, protein array, and RNA-seq data. The article reviews its statistical design and recent extensions, including voom-based RNA-seq analysis, differential splicing, and higher-level gene-set/signature analyses.

Key findings

  • limma now performs both differential expression and differential splicing for RNA-seq data, with all downstream tools previously restricted to microarrays available for RNA-seq; the same pipeline can be used for RNA-seq and microarray data after preprocessing.
  • Empirical Bayes methods borrow information between genes to moderate variances, improving inference in small-sample experiments; recent enhancements include a mean-variance trend and robust empirical Bayes with gene-wise shrinkage factors.
  • Higher-level analyses include gene-set testing functions such as `goana`, `geneSetTest`, `camera`, `roast`, and `romer`, plus `genas` for estimating biological correlation between contrasts and `propTrueNull` for estimating the proportion of true null hypotheses.
  • The package is freely available in Bioconductor; more than 120 other Bioconductor packages made use of limma as of March 2014, and the user’s guide is over 120 pages with 10 fully worked case studies.

Methods

  • This is a review/software description article, not a primary experimental study; it summarizes limma’s philosophy, design, and features, with example diagnostic and DE plots generated from previously published datasets.
  • Example data sources include RNA-seq data from Pickrell et al. analysed as described by Law et al., two-colour microarray quality-control data from Ritchie et al., two-colour microarray data from GEO series GSE2593, and RNA-seq data from GEO series GSE52870 analysed as described in Liu et al.
  • The core analysis framework uses gene-wise linear models, design matrices, contrasts, empirical Bayes moderated t- and F-statistics, quantitative quality weights, `duplicateCorrelation` for blocking/random effects, and `voom` precision weights for RNA-seq counts.

Limitations

  • The paper states that limma does not perform permutation or re-sampling-based inference and discusses disadvantages of permutation for complex designs, small replicate numbers, correlated samples, and unequal precision.
  • The paper states that parametric distributions “can never be perfectly correct” but argues that modelling the variance structure more realistically outweighs these compromises in small, complex experiments.
  • Evident from the text: the article is a software review summarizing existing and recent features, not a new empirical benchmark; performance claims are largely qualitative or cite prior work, and no new head-to-head numerical comparisons are reported in this article.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/nar/gkv007

Cites

72 of 72 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 61 cited sources checked · 0 supported · 0 flagged

  • Findingnone given

    limma incorporates observation weights to allow for variations in data quality.

    “(ii) incorporate observation weights (wgj where j refers to sample) to allow for variations in data quality”
    Quote in the paperNot a substantial claim
  • Findingnone given

    limma can go past traditional gene-wise expression analyses by analysing expression profiles in terms of co-regulated sets of genes or higher-order expression signatures.

    “Second, the package is now able to go past the traditional gene-wise expression analyses in a variety of ways, analysing expression profiles in terms of co-regulated sets of genes or in terms of higher-order expression signatures.”
    Quote in the paperSubstantial claim
  • Findingnone given

    limma's empirical Bayes methods facilitate information borrowing to obtain posterior variance estimators.

    “The limma package includes statistical methods that (i) facilitate information borrowing using empirical Bayes methods to obtain posterior variance estimators”
    Quote in the paperSubstantial claim
  • Findingnone given

    The hallmark of limma is the use of linear models to analyse entire experiments as an integrated whole rather than making piece-meal comparisons between pairs of treatments.

    “The hallmark of the limma approach is the use of linear models to analyse entire experiments as an integrated whole rather than making piece-meal comparisons between pairs of treatments.”
    Quote in the paperSubstantial claim
  • Findingreasoning

    limma allows users to analyse both RNA-seq and microarray data with very similar pipelines.

    “These capabilities allow users to analyse both RNA-seq and microarray data with very similar pipelines.”
    Quote in the paperSubstantial claim
  • May combine two claims. 41
  • Classified as probably not a substantial claim. 58
  • The quote only partly states this claim. 1
  • May combine two claims.

    Findingnone given

    limma contains features for handling complex experimental designs and for information borrowing to overcome small sample sizes.

    “It contains rich features for handling complex experimental designs and for information borrowing to overcome the problem of small sample sizes.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Gene expression studies often involve only a small number of biological replicates.
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    All features of limma's statistical models can be accessed for gene-wise expression analyses and for higher level analyses of gene expression signatures.

    “All the features of the statistical models can be accessed not just for gene-wise expression analyses but also for higher level analyses of gene expression signatures.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Findingnone given

    limma is designed so that, after initial pre-processing and normalization, the same analysis pipeline is used for data from all technologies.

    “The package is designed in such a way that, after initial pre-processing and normalization, the same analysis pipeline is used for data from all technologies.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    limma has proven a popular choice for analysis of data from experiments involving microarrays, high-throughput PCR, protein arrays and other platforms.

    “It has proven a popular choice for the analysis of data from experiments involving microarrays (3,4), high-throughput polymerase chain reaction (PCR) (5), protein arrays (6) and other platforms.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [3] Peart, 2005, Identification and functional significance of genes regulated by structurally different histone deacetylase inhibitors, Proc. Natl. Acad. Sci. U.S.A. source

      Not checked in the free run.

    • cites [4] Caiazzo, 2011, Direct generation of functional dopaminergic neurons from mouse and human fibroblasts, Nature source

      Not checked in the free run.

    • cites [5] Hubert, 2009, Aire-deficient c57bl/6 mice mimicking the common human 13-base pair deletion mutation present with only a mild autoimmune phenotype, J. Immunol. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    limma can perform both differential expression and differential splicing analyses of RNA sequencing (RNA-seq) data.

    “First, the package can now perform both differential expression and differential splicing analyses of RNA sequencing (RNA-seq) data.”
    • Also stated in: § INTRODUCTION · plain
      “First, the package can now perform both differential expression (DE) and differential splicing analyses of RNA sequencing (RNA-seq) data (7,8).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [7] Liu, 2014, Pax5 loss imposes a reversible differentiation block in B progenitor acute lymphoblastic leukemia, Genes Dev. source

      Not checked in the free run.

    • cites [8] Su, 2014, A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium, Nat. Biotechnol. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    limma includes pre-processing methods such as variance stabilization to reduce noise.

    “(iv) pre-processing methods such as variance stabilization to reduce noise.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingreasoning

    The voom function converts the mean-variance trend into precision weights, which are incorporated into the analysis of log-transformed RNA-seq counts using the same linear modelling commands as for microarrays.

    “The mean-variance trend is converted by the voom function into precision weights, which are incorporated into the analysis of log-transformed RNA-seq counts using the same linear modelling commands as for microarrays.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Output in other formats can be read if the appropriate column names are supplied.

    “Output in other formats can be read if the appropriate column names are supplied.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingnone given

    Weights can be used to give more emphasis to control probes during normalization, or to down-weight less reliable measurements or samples.

    “Weights can be used to give more emphasis to control probes during normalization, or can be used to down-weight measurements or samples that are less reliable in a gene expression analysis.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    read.maimages directly supports formats written by many different image analysis programs including GenePix, Agilent Feature Extraction, ArrayVision, BlueFuse, ImaGene, QuantArray and SPOT.

    “This function directly supports formats written by many different image analysis programs including GenePix, Agilent Feature Extraction, ArrayVision, BlueFuse, ImaGene, QuantArray and SPOT (Table 1).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    The logCPM values can be normalized between samples by the voom function or can be pre-normalized by adding normalization factors within edgeR.

    “The logCPM values can be normalized between samples by the voom function or can be pre-normalized by adding normalization factors within edgeR.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Raw read counts are assembled outside limma using tools such as featureCounts, HTSeq-counts or RSEM.

    “Raw read counts are assembled outside limma using tools such as featureCounts (29), HTSeq-counts (30) or RSEM (31).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [29] Liao, 2014, featureCounts: an efficient general-purpose read summarization program, Bioinformatics source

      Not checked in the free run.

    • cites [30] Anders, 2015, HTSeq—a Python framework to work with high-throughput sequencing data, Bioinformatics source

      Not checked in the free run.

    • cites [31] Li, 2011, RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome, BMC Bioinformatics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    read.maimages also supports the Stanford Microarray Database format.

    “It also supports the Stanford Microarray Database format.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    These ideas were part of the original limma package submitted to Bioconductor in 2003.

    “These ideas were part of the original limma package submitted to Bioconductor in 2003.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingnone given

    Linear models permit very general analyses, including adjusting for multiple experimental factors, batch effects, time course effects, regression splines, gene expression covariates, and testing interaction effects or complex customized comparisons.

    “Linear models permit very general analyses. Researchers can adjust for the effects of multiple experimental factors or can adjust for batch effects. The linear model might include time course effects or regression splines. The linear model could even include the expression values themselves of one or more genes as covariates, allowing researchers to test for inter-gene dependencies. Linear models”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingnone given

    Weights can be applied to genes, RNA samples, or individual expression values.

    “Weights can be applied to genes or to RNA samples or to individual expression values.”
    Quote in the paperSubstantial claim
  • May combine two claims. The quote only partly states this claim.

    Findingreasoning

    limma generalized the concept of an MA-plot to apply to sets of single-channel expression values and to fitted model objects.

    “limma generalized the concept of an MA-plot in two ways. First, the idea was extended to apply to sets of single-channel expression values. In this case, the plot is used to compare each sample to the average of all other samples. A virtual array is constructed by averaging the log-expression value for all the samples other than the sample of interest, and then a mean-difference plot is made”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingnone given

    limma implements global covariance models to estimate correlations between genes or the relatedness between the DE profiles resulting from different comparisons.

    “A closely related statistical approach implemented in limma is to fit global covariance models, either to estimate correlations between genes or to estimate the relatedness between the DE profiles resulting from difference comparisons.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning

    Two-colour and single-channel data are both supported.

    “Two-colour and single-channel data are both supported.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    The nec function implements normexp background correction for Illumina BeadChips using control probes specific to these arrays.

    “The nec function implements normexp background correction for Illumina BeadChips making special use of the control probes that are specific to these arrays (24).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [24] Shi, 2010, Optimizing the noise versus bias trade-off for Illumina Whole Genome Expression Beadchips, Nucleic Acids Res. source

      Not checked in the free run.

  • May combine two claims.

    Findingreasoning

    This strategy allows gene-wise models to incorporate correlations between duplicate probes, correlations between related RNA samples, or variations in quality between RNA samples.

    “This strategy allows the gene-wise models to incorporate such things as correlations between duplicate probes for the same gene, or correlations between related RNA samples, or variations in quality between the RNA samples.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingcitation

    limma includes a maximum likelihood implementation of the normal-exponential convolution model for background correction and an implementation of loess curves and normalization using quantitative weights.

    “Notable are the maximum likelihood implementation of the normal-exponential convolution model for background correction (22) and the implementation of loess curves and normalization using quantitative weights.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] Silver, 2009, Microarray background correction: maximum likelihood estimation for the normal–exponential convolution, Biostatistics source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    One way is through estimating a mean-variance trend, which can either be incorporated into the empirical Bayes procedure or used to generate observation weights.

    “One way is through estimating a mean-variance trend, which can either be incorporated into the empirical Bayes procedure as mentioned above or used to generate observation weights (10).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [10] Law, 2014, Voom: precision weights unlock linear model analysis tools for RNA-seq read counts, Genome Biol. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The main limma function to read image output files is read.maimages.

    “The main limma function to read image output files is read.maimages.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingnone given

    A separate model is fitted for each gene, but the gene-wise models can be linked by global parameters or global hyper-parameters.

    “A separate model is fitted for each gene, but the gene-wise models can be linked by global parameters or global hyper-parameters.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The voom function converts read counts into log2 counts per million with associated precision weights.

    “The read counts are processed by the voom function in limma to convert them into log2 counts per million (logCPM) with associated precision weights.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingreasoning

    Use of global parameters shares information between genes and can be used even for the smallest experiments because the global parameters can be estimated from the entire data set.

    “The use of global parameters is a simple means of sharing information between genes that can be used even for the smallest experiments, because the global parameters can be estimated from the entire data set involving all the genes at once.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingcitation

    This allows a robust empirical Bayes procedure in which hyper-variable genes are identified and treated separately.

    “This allows a sophisticated robust empirical Bayes procedure in which hyper-variable genes are identified and treated separately (18,19).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [18] Phipson, 2013, Empirical Bayes in the presence of exceptional cases, with application to microarray data

      Not checked in the free run.

    • cites [19] Phipson, 2013, Empirical Bayes modelling of expression profiles and their associations

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    RNA-seq data can be explored using boxplots or mean-difference plots, similarly to single-channel microarray data.

    “For example, RNA-seq data can be explored using boxplots or mean-difference plots, similarly to single-channel microarray data.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    Illumina BeadChips need special treatment: output from Illumina's GenomeStudio can be read by read.ilmn if exported as a text file or by read.idat if in binary format.

    “Illumina BeadChips need special treatment: output from Illumina's GenomeStudio can be read by read.ilmn if exported as a text file or by read.idat if in binary format.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    limma can accept a DGEList object from the edgeR package.

    “Alternatively it can accept a DGEList object from the edgeR package.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingcitation

    Use of weights increases power to detect differentially expressed genes, and a model-based approach avoids ad hoc decisions about which observations or samples to filter out.

    “The use of weights increases power to detect differentially expressed genes, and having a model based approach avoids the need for ad hoc decisions about which observations or samples to filter out (11).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [11] Ritchie, 2006, Empirical array quality weights in the analysis of microarray data, BMC Bioinformatics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    limma accepts RNA-seq data in the form of a matrix of read counts, with rows for genomic features and columns for RNA samples.

    “limma accepts RNA-seq data in the form of a matrix of read counts, with rows for genomic features and columns for RNA samples.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingcitation

    The resulting pipeline gives comparable performance to the best negative binomial-based software packages but with greater speed and reliability for large data sets.

    “The resulting pipeline gives comparable performance to the best of the negative binomial-based software packages but with greater speed and reliability for large data sets (10,21).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [10] Law, 2014, Voom: precision weights unlock linear model analysis tools for RNA-seq read counts, Genome Biol. source

      Not checked in the free run.

    • cites [21] Rapaport, 2013, Comprehensive evaluation of differential gene expression analysis methods for RNA-seq data, Genome Biol. source

      Not checked in the free run.

  • May combine two claims.

    Findingreasoning

    Analysing data as a whole shares information between samples and allows modelling correlations between samples due to repeated measures or other causes.

    “This has the effect of sharing information between samples. Analysing the data as a whole also allows us to model correlations that may exist between samples due to repeated measures or other causes.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingnone given

    Weights can be preset based on external quality information or estimated from the expression data itself.

    “The weights can be preset based on external quality information, or may be estimated from the expression data itself.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    limma is the only software package to provide methods for error rate control across multiple contrasts and genes simultaneously.

    “limma is the only software package to provide methods for error rate control across multiple contrasts and genes simultaneously.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Figure 3A shows that total variability asymptotes to biological variability as count sizes increase.

    “Total variability asymptotes to biological variability as count sizes increases.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    The propexpr function estimates the total proportion of probes on each array that correspond to expressed genes, providing an estimate of transcriptome size.

    “The propexpr function compares intensities to those of negative control probes to estimate the total proportion of probes on each array that correspond to expressed genes (34).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] Shi, 2010, Estimating the proportion of microarray probes expressed in an RNA sample, Nucleic Acids Res. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Many limma functions are generic or operate appropriately on objects of different classes.

    “Many limma functions are generic or operate appropriately on objects of different classes.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingnone given

    limma provides the possibility of analysing two-colour microarrays as if they were single channel microarrays with two separate samples hybridized to each physical array, allowing intensities to be directly compared between microarrays.

    “limma also provides the possibility of analysing two-colour microarrays as if they were single channel microarrays with two separate samples hybridized to each physical array. This provides a very powerful type of analysis in which intensities can be directly compared between microarrays.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingcitation

    The treat function tests whether the log2-fold-change is greater than a threshold rather than merely different to zero, which can be effective for prioritizing biologically as well as statistically significant results.

    “the treat function can be used to test whether the log2-fold-change is greater than a threshold rather than merely different to zero (45). This can be effective for prioritizing results that are biologically as well as statistically significant.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [45] McCarthy, 2009, Testing significance relative to a fold-change threshold is a TREAT, Bioinformatics source

      Not checked in the free run.

  • May combine two claims.

    Findingreasoning

    All linear model fits in limma produce a fitted model object with the same structure, allowing the same framework for DE to be used for all experimental designs and platform technologies.

    “All the linear model fits, whether using lmFit or lmscFit, produce a fitted model object with the same structure.”
    • Also stated in: § Finding differentially expressed genes · plain
      “This consistency allows the same framework for DE to be used for all experimental designs and platform technologies.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    The neqc function implements quantile normalization for Illumina BeadChips using control probes specific to these arrays.

    “The neqc function implements quantile normalization for Illumina BeadChips making special use of the control probes specific to these arrays (24).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [24] Shi, 2010, Optimizing the noise versus bias trade-off for Illumina Whole Genome Expression Beadchips, Nucleic Acids Res. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Figure 4C shows a strong positive concordance between Pax5 restoration and the large to small cell transition during normal B cell development.

    “The plot shows a strong positive concordance between Pax5 restoration and the large to small cell transition.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [7] Liu, 2014, Pax5 loss imposes a reversible differentiation block in B progenitor acute lymphoblastic leukemia, Genes Dev. source

      Not checked in the free run.

    • cites [47] Hoffmann, 2002, Changes in gene expression profiles in developing b cells of murine bone marrow, Genome Res. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    The memory requirements of limma are linear in the number of genes and the number of samples.

    “The memory requirements are linear in the number of genes and the number of samples.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Barcodeplot is similar to the set location plot introduced by Subramanian et al., with enhancements including the ability to incorporate genes with positive and negative prior directions.

    “Barcodeplot is similar to the set location plot introduced by Subramanian et al. (63), with a number of enhancements, especially the ability to incorporate genes with positive and negative prior directions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [63] Subramanian, 2005, Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles, Proc. Natl. Acad. Sci. U.S.A. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The propTrueNull function estimates the proportion of true null hypotheses in a collection of hypothesis tests given a vector of P-values.

    “The propTrueNull function estimates the number of truly differentially expressed genes that remain to be identified. Mathematically, it estimates the proportion of true null hypotheses in a collection of hypothesis tests given a vector of P-values.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The romer function in limma implements a GSEA approach that is based on rotation instead of permutation.

    “The romer function in limma implements a GSEA approach that is based on rotation instead of permutation.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    The romer function can be used with a battery of gene sets and with any linear model.

    “Like camera and mroast, it can be used with a battery of gene sets and with any linear model.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingcitation

    limma includes a robustified shrinkage strategy that allows gene-wise shrinkage factors to be estimated, ensuring unusually large variances are not squeezed too heavily, reducing the chance they appear statistically significant, while more consistently expressed genes are squeezed more severely.

    “limma includes a robustified shrinkage strategy that allows for gene-wise shrinkage factors to be estimated (18). This ensures unusually large variances are not squeezed too heavily, reducing the chance that they will appear statistically significant, while more consistently expressed genes are squeezed more severely towards the common variance.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [18] Phipson, 2013, Empirical Bayes in the presence of exceptional cases, with application to microarray data

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Figure 3A shows that technical variability decreases with count size for RNA-seq data.

    “This plot shows that technical variability decreases with count size.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    The limma authors maintain mouse and human versions of the Molecular Signatures Database collections in R binary format that can be conveniently used with camera, mroast or romer.

    “The limma authors maintain mouse and human versions of the Molecular Signatures Database collections (64) in R binary format that can be conveniently used with camera, mroast or romer”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [64] Liberzon, 2011, Molecular signatures database (MSigDB) 3.0, Bioinformatics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The limma software is freely available online as part of the Bioconductor project.

    “The limma software is freely available online as part of the Bioconductor project (http://www.bioconductor.org).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    Most estimation procedures in limma finish in a few seconds on a standard desktop computer and virtually all in less than a minute.

    “Most estimation procedures finish in a few seconds on a standard desktop computer and virtually all in less than a minute.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    The geneSetTest and wilcoxGST functions perform rank-based tests that assess whether a specified set of genes is more highly ranked in an ordered list of all genes than would be expected by chance.

    “The simplest approach is implemented in the geneSetTest and wilcoxGST functions, which perform rank-based tests (13). These tests assess whether the specified set of genes is more highly ranked in an ordered list of all genes than would be expected by chance.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] Michaud, 2008, Integrative analysis of runx1 downstream pathways and target genes, BMC Genomics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Each function in limma has its own documentation page that specifies the input data, options and output format of the function.

    “First, each function has its own documentation page that concisely but completely specifies the input data, options and output format of the function.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingreasoning

    All of the data classes in limma obey many analogies with matrices.

    “All of these data classes obey many analogies with matrices.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Camera has been used successfully in a number of biomedical projects.

    “Camera has been used successfully in a number of biomedical projects (57).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [57] Wu, 2013, Gene-expression data integration to squamous cell lung cancer subtypes reveals drug sensitivity, Br. J. Cancer source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    The plotMDS function uses multi-dimensional scaling to plot differences in expression profiles between samples, with distances representing 'leading fold change' defined as the root-mean-square average of log-fold-changes for the genes best distinguishing each pair of samples.

    “The plotMDS function uses multi-dimensional scaling to plot differences in expression profiles between different samples (Figure 3C). Distances between samples on the plot represent ‘leading fold change’, which is defined as the root-mean-square average of the log-fold-changes for the genes best distinguishing each pair of samples.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    All between-arrays normalization methods are accessible for RNA-seq data from within the voom function.

    “All the between-arrays normalization methods are accessible for RNA-seq data from within the voom function.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    More than 120 other Bioconductor packages make use of limma as of March 2014.

    “More than 120 other Bioconductor packages make use of limma (as of March 2014).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingnone given

    limma includes enhancements: ability to replace the loess curve with a spline curve with high robustness breakdown properties, and ability to apply empirical Bayes moderation to spline curves for multiple regions within the same array.

    “Other enhancements include the ability to replace the loess curve with a spline curve that has high robustness breakdown properties, and the ability to apply empirical Bayes moderation to the spline curves for multiple regions within the same array (robust-spline normalization).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingcitation

    The camera function computes a variance inflation factor from the inter-gene correlation and uses this to adjust the variance of the summary statistics, avoiding optimistic P-values but reducing statistical power.

    “Camera computes a variance inflation factor from the inter-gene correlation and uses this to adjust the variance of the summary statistics. This avoids optimistic P-values in the test results, but also reduces statistical power.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [56] Wu, 2012, Camera: a competitive gene set test accounting for inter-gene correlation, Nucleic Acids Res. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    The limma package is implemented primarily in R and includes some C code to speed up computationally intensive steps.

    “The limma package is implemented primarily in R (65) and includes some C code to speed up computationally intensive steps.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [65] R Development Core Team, 2014

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    The limma package provides three levels of documentation.

    “The limma package provides three levels of documentation.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    limma defines a number of classes that have been tailored to handle both microarray and RNA-seq data.

    “limma defines a number of classes that have been tailored to handle both microarray and RNA-seq data.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Applications of limma's linear modelling strategy beyond the intended analysis of gene expression data have been made in a variety of applications, including the analysis of data from Nuclear Magnetic Resonance spectroscopy, PCR (including Nanostring), quantitative proteomics, DNA methylation arrays and comparative ChIP-seq.

    “Applications of limma's linear modelling strategy beyond the intended analysis of gene expression data have been made in a variety of applications, including the analysis of data from Nuclear Magnetic Resonance spectroscopy, PCR (including Nanostring), quantitative proteomics (80), DNA methylation arrays and comparative ChIP-seq (81).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [80] Brusniak, 2008, Corra: computational framework and tools for LC-MS discovery and targeted mass spectrometry-based proteomics, BMC Bioinformatics source

      Not checked in the free run.

    • cites [81] Lun, 2014, De novo detection of differentially bound regions for ChIP-seq data using peaks and windows: controlling error rates correctly, Nucleic Acids Res. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Each data class in limma has a documentation page explaining all the required and optional components of objects of that class.

    “Similarly, each data class has a documentation page explaining all the required and optional components of objects of that class.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The limma package comes with an extensive user's guide of over 120 pages.

    “Third, the package comes with an extensive user's guide of over 120 pages, available from the drop-down menu in Windows or alternatively launched by the limmaUsersGuide command.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The limma user's guide includes 10 fully worked case studies for which full data and code are provided.

    “It also includes 10 fully worked case studies for which full data and code are provided.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    The Bioconductor support site archives answers to many common questions, including many queries about experimental design and setting up appropriate design matrices.

    “The support site archives answers to many common questions, including many queries about experimental design and setting up appropriate design matrices.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    A series of more general subject help pages in limma link together functions and classes that are used for related purposes.

    “Second, a series of more general subject help pages serve to link together functions and classes that are used for related purposes.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The limma user's guide gives detailed advice on how to analyse a variety of common study designs.

    “The user's guide gives detailed advice on how to analyse a variety of common study designs.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    limma is a core component of Bioconductor.

    “The limma package is a core component of Bioconductor, an R-based open-source software development project in statistical genomics (1,2).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [1] Gentleman, 2004, Bioconductor: open software development for computational biology and bioinformatics, Genome Biol. source

      Not checked in the free run.

    • cites [2] Smyth, 2005, Limma: linear models for microarray data source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationcitation

    Simply subtracting background from foreground intensities is too heavy-handed.

    “Simply subtracting background from foreground intensities is too heavy-handed (23).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] Ritchie, 2007, A comparison of background correction methods for two-colour microarrays, Bioinformatics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    limma's backgroundCorrect function offers a range of more sophisticated alternatives to simple background subtraction, most unique to the package.

    “The limma backgroundCorrect function offers a range of more sophisticated alternatives, most unique to the package.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Premisecitation

    Traditionally, RNA-seq data require specialized software based on the negative binomial or similar distributions.

    “Traditionally, RNA-seq data require specialized software based on the negative binomial or similar distributions (20).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [20] Robinson, 2010, edgeR: a Bioconductor package for differential expression analysis of digital gene expression data, Bioinformatics source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    If permutation is applied to all samples in a multi-factor experiment, the composite null hypothesis being tested is uninteresting and the power to reject it may be highly dependent on the existence of differential expression between treatment conditions that are not of primary interest.

    “If permutation is applied to all the samples in a multi-factor experiment, then the composite null hypothesis being testing is an uninteresting one and the power to reject it may be highly dependent on the existence of DE between treatment conditions that are not of primary interest.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The plotMDS plot provides unsupervised clustering of samples and is useful for identifying unexpected patterns such as batch effects that should be adjusted for.

    “This provides a type of unsupervised clustering of the samples. It is useful for examining how different are the profiles produced by different experimental factors and for identifying unexpected patterns, such as batch effects, that should be adjusted for during the linear model analysis.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    The ability to use quantitative weights in loess normalization allows downweighing less reliable probes or giving higher priority to control probes or house-keeping genes.

    “giving it the ability to downweigh less reliable probes or to give higher priority to control probes or house-keeping genes.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationnone given

    The flexibility of the linear modelling approach allows almost any experimental design to be handled, including experiments with two or more groups, factorial and time-course designs, and internal controls such as dye-swaps.

    “The flexibility of the linear modelling approach allows almost any experimental design to be handled. Experiments with two or more groups, factorial and time-course designs, and internal controls such as dye-swaps can all be modelled and summarized using the lmFit function.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    The robust strategy offers the benefits of shrinkage to the majority of genes while negating the effects of outliers.

    “This robust strategy offers the benefits of shrinkage to the majority of the genes, whilst negating the effects of outliers.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    The empirical Bayes framework reduces the number of false positives for genes with very small variances and improves power to detect DE for genes with larger variances.

    “which reduces the number of false positives for genes with very small variances and improves power to detect DE for genes with larger variances.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    The differential splicing approach is considerably faster than alternative approaches, making large-scale surveys of differential exon usage feasible.

    “This approach is considerably faster than alternative approaches to differential splicing, making large-scale surveys of differential exon usage feasible.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Different technology platforms introduce different biases and so require different normalization methods.

    “Different technology platforms introduce different biases and so require different normalization methods.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationcitation

    These rank-based tests implicitly assume that the expression level of each gene is conditionally independent of other genes and hence give optimistic P-values.

    “but they implicitly assume that the expression level of each gene is conditionally independent of other genes and hence give optimistic P-values (55).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [55] Wu, 2010, ROAST: rotation gene set tests for complex microarray experiments, Bioinformatics source

      Not checked in the free run.

    • The authors state this as an assumption: The geneSetTest and wilcoxGST rank-based tests assume that the expression level of each gene is conditionally independent of other genes.
  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    The development of the voom methodology unlocks the majority of analysis methods for use on RNA-seq data, such as random effects modelling and gene set testing.

    “Although originally developed with microarray data in mind, the development of the voom methodology unlocks the majority of analysis methods for use on RNA-seq data, such as random effects modelling and gene set testing.”
    Quote in the paperNot a substantial claim
88 of 88
  • The paper relies on this without support.

    Gene expression studies often involve only a small number of biological replicates.

    “Despite the complexity, gene expression studies often involve only a small number of biological replicates.”
  • The paper relies on this without support.

    Many gene expression technologies produce data that are less reliable at lower intensities or abundances.

    “This is important because many gene expression technologies produce data that are less reliable at lower intensities or abundances.”
  • The paper relies on this without support.

    Permutation assumes that all samples are independent and identically distributed under the null hypothesis, and these assumptions are frequently unrealistic.

    “Even more importantly, permutation assumes that all samples are independent and identically distributed under the null hypothesis, and these assumptions are frequently, usually perhaps, unrealistic.”
  • The authors state this as an assumption.

    The geneSetTest and wilcoxGST rank-based tests assume that the expression level of each gene is conditionally independent of other genes.

    “but they implicitly assume that the expression level of each gene is conditionally independent of other genes”
  • The paper relies on this without support.

    Most gene expression studies have false negatives: truly differentially expressed genes that are not detected because the study did not have enough statistical power.

    “However, for most studies, there are likely to be false negatives: truly differentially expressed genes that are not detected as differentially expressed because the study did not have enough statistical power to identify them with confidence.”
  • The paper relies on this without support.

    Gene symbols change over time.

    “Gene symbols change over time”
  • The paper relies on this without support.

    The use of gene sets requires that gene symbols and annotation be matched between different databases and studies.

    “The use of gene sets require that gene symbols and annotation be matched between different databases and studies.”
7 of 7

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Conflicting name for the foreground-background plotting function

    Minor

    The background-correction section names the function plotFB, but the diagnostic-plots section calls the foreground-background plot mentioned above plotFG.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 10

  1. Efron, 1973, Stein's estimation rule and its competitors—an empirical Bayes approach, J. Am. Stat. Assoc., 68, 117
  2. Phipson, 2013, Empirical Bayes in the presence of exceptional cases, with application to microarray data
  3. Phipson, 2013, Empirical Bayes modelling of expression profiles and their associations
  4. Cleveland, 1993, Visualizing Data
  5. Dudoit, 2002, Statistical methods for identifying differentially expressed genes in replicated cDNA microarray experiments, Stat. Sin., 12, 111
  6. Liao, 2014, featureCounts: an efficient general-purpose read summarization program, Bioinformatics, 30, 923, 10.1093/bioinformatics/btt656 · DOI · Paper page
  7. Ritchie, 2004, Quantitative quality control and background correction for two-colour microarray data
  8. R Development Core Team, 2014, R: A Language and Environment for Statistical Computing
  9. Leisch, 2002, Sweave: dynamic generation of statistical reports using literate data analysis, 575
  10. Xie, 2013, Dynamic Documents with R and knitr

Cited by

Sorted by citations

Authors
Ritchie, M. E., Phipson, B., Wu, D., Hu, Y., Law, C. W., Shi, W., & Smyth, G. K.
Affiliations
  • The University of Melbourne (AU)
  • Royal Children's Hospital (AU)
  • Harvard University (US)
  • Walter and Eliza Hall Institute of Medical Research (AU)
  • University of Zurich (CH)
Corresponding author
Smyth, Gordon K.
Venue
Nucleic Acids Research
Published
2015-01-20
Volume
43
Issue
7
Pages
e47-e47
ISSN
  • 0305-1048 (ISSN-L)
  • 1362-4962
OpenAlex
W2146512944
Cited by
40,458
References
72
Citations by year
2026: 4544 · 2025: 5451 · 2024: 5748 · 2023: 5274 · 2022: 5276 · 2021: 4686 · 2020: 3473 · 2019: 2299 · 2018: 1588 · 2017: 1200 · 2016: 684 · 2015: 206 · 2014: 1 · 2013: 1 · 2012: 2
FWCI
532.21
Type
journal article
Language
en
Open access
Locations
  • Nucleic Acids Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by)
  • reroDoc Digital Library (submittedVersion)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by)
  • Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
  • Figshare (submittedVersion, cc-by)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
  • La Trobe University (cc-by)
  • Zurich Open Repository and Archive (University of Zurich)
Funders
  • Australian Government
  • Medical Research Council
  • National Health and Medical Research Council (1054618, 1050661, 1023454)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
Keywords
limma, differential expression analysis, RNA-seq, microarray data, differential splicing, gene expression normalization, information borrowing, small sample size, gene expression signatures, Bioconductor, R software
MeSH terms
Gene Expression Regulation, Software, Sequence Analysis, RNA, Oligonucleotide Array Sequence Analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review