CoChat

Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures

Open access
Munro, Sarah A., Lund, Steven P., Pine, P. Scott, Binder, Hans, Clevert, Djork-Arné, Conesa, Ana, Dopazo, Joaquin, Fasold, Mario, Hochreiter, Sepp, Hong, Huixiao, Jafari, Nadereh, Kreil, David P., Łabaj, Paweł P., Li, Sheng, Liao, Yang, Lin, Simon M., Meehan, Joseph, Mason, Christopher E., Santoyo-Lopez, Javier, Setterquist, Robert A., Shi, Leming, Shi, Wei, Smyth, Gordon K., Stralis-Pavese, Nancy, Su, Zhenqiang, Tong, Weida, Wang, Charles, Wang, Jian, Xu, Joshua, Ye, Zhan, Yang, Yong, Yu, Ying, Salit, Marc · 2014 · Nature Communications
Cited by 124

The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.

Abstract

There is a critical need for standard approaches to assess, report and compare the technical performance of genome-scale differential gene expression experiments. Here we assess technical performance with a proposed standard ‘dashboard’ of metrics derived from analysis of external spike-in RNA control ratio mixtures. These control ratio mixtures with defined abundance ratios enable assessment of diagnostic performance of differentially expressed transcript lists, limit of detection of ratio (LODR) estimates and expression ratio variability and measurement bias. The performance metrics suite is applicable to analysis of a typical experiment, and here we also apply these metrics to evaluate technical performance among laboratories. An interlaboratory study using identical samples shared among 12 laboratories with three different measurement processes demonstrates generally consistent diagnostic power across 11 laboratories. Ratio measurement variability and bias are also comparable among laboratories for the same measurement process. We observe different biases for measurement processes using different mRNA-enrichment protocols. Differential gene expression experiments yield quantitative insight into biological activity and may be important in disease classification and treatment. Here, the authors analyse external spike-in RNA controls to provide a standard method to assess and compare experiment performance.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/ncomms6125

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 13 references

Research works printed but not linked: 23

  1. Salit, M. Standards in gene expression microarray experiments. Methods Enzymol. 411, 63–78 (2006). · DOI
  2. Lippa, K. A., Duewer, D. L., Salit, M. L., Game, L. & Causton, H. C. Exploring the use of internal and external controls for assessing microarray technical performance. BMC Res. Notes 3, 349 (2010). · DOI
  3. McCall, M. N. & Irizarry, R. A. Consolidated strategy for the analysis of microarray spike-in data. Nucleic Acids Res. 36, e108 (2008). · DOI
  4. Tong, W. et al. Evaluation of external RNA controls for the assessment of microarray performance. Nat. Biotechnol. 24, 1132–1139 (2006). · DOI
  5. van de Peppel, J. et al. Monitoring global messenger RNA changes in externally controlled microarray experiments. EMBO Rep. 4, 387–393 (2003). · DOI · Paper page
  6. Adiconis, X. et al. Comparative analysis of RNA sequencing methods for degraded or low-input samples. Nat. methods 10, 623–629 (2013). · DOI · Paper page
  7. Gertz, J. et al. Transposase mediated construction of RNA-seq libraries. Genome Res. 22, 134–141 (2012). · DOI
  8. Griffith, M. et al. Alternative expression analysis by RNA sequencing. Nat. Methods 7, 843–847 (2010). · DOI · Paper page
  9. Ramskold, D. et al. Full-length mRNA-Seq from single-cell levels of RNA and individual circulating tumor cells. Nat. Biotechnol. 30, 777–782 (2012). · DOI · Paper page
  10. Novoradovskaya, N. et al. Universal Reference RNA as a standard for microarray experiments. BMC Genomics 5, 20 (2004). · DOI · Paper page
  11. Pine, P. S. et al. Use of diagnostic accuracy as a metric for evaluating laboratory proficiency with microarray assays using mixed-tissue RNA reference samples. Pharmacogenomics 9, 1753–1763 (2008). · DOI
  12. Dudoit, S., Yang, Y. H., Callow, M. J. & Speed, T. P. Statistical methods for identifying differentially expressed genes in replicated cDNA microarray experiments. Stat. Sin. 12, 111–139 (2002).
  13. Liao, Y., Smyth, G. K. & Shi, W. The Subread aligner: fast, accurate and scalable read mapping by seed-and-vote. Nucleic Acids Res. 41, e108 (2013). · DOI · Paper page
  14. Liao, Y., Smyth, G. K. & Shi, W. featureCounts: an efficient general purpose program for assigning sequence reads to genomic features. Bioinformatics 30, 923–930 (2014). · DOI · Paper page
  15. Trapnell, C., Pachter, L. & Salzberg, S. L. TopHat: discovering splice junctions with RNA-Seq. Bioinformatics 25, 1105–1111 (2009). · DOI · Paper page
  16. Li, H. & Durbin, R. Fast and accurate short read alignment with Burrows-Wheeler transform. Bioinformatics 25, 1754–1760 (2009). · DOI · Paper page
  17. Fasold, M., Stadler, P. F. & Binder, H. G-stack modulated probe intensities on expression arrays—sequence corrections and signal calibration. BMC Bioinformatics 11, (2010). · DOI
  18. Hochreiter, S., Clevert, D. A. & Obermayer, K. A new summarization method for affymetrix probe level data. Bioinformatics 22, 943–949 (2006). · DOI · Paper page
  19. Mueckstein, U., Leparc, G. G., Posekany, A., Hofacker, I. & Kreil, D. P. Hybridization thermodynamics of NimbleGen microarrays. BMC Bioinformatics 11, 35 (2010). · DOI
  20. Gentleman, R. C. et al. Bioconductor: open software development for computational biology and bioinformatics. Genome Biol. 5, R80 (2004). · DOI · Paper page
  21. Lund, S., Nettleton, D., McCarthy, D. & Smyth, G. Detecting differential expression in RNA-sequence data using quasi-likelihood with shrunken dispersion estimates. Stat. Appl. Genet. Mol. Biol. 11,, Iss. 5, Art. 8 (2012). · DOI · Paper page
  22. Robinson, M. D., McCarthy, D. J. & Smyth, G. K. edgeR: a Bioconductor package for differential expression analysis of digital gene expression data. Bioinformatics 26, 139–140 (2010). · DOI · Paper page
  23. Wickham, H. ggplot2: Elegant Graphics for Data Analysis Springer (2009). · DOI · Paper page

Cited by

See all 124

Metadata

Authors
Munro, S. A., Lund, S. P., Pine, P. S., Binder, H., Clevert, D.-A., Conesa, A., Dopazo, J., Fasold, M., Hochreiter, S., Hong, H., Jafari, N., Kreil, D. P., Łabaj, P. P., Li, S., Liao, Y., Lin, S. M., Meehan, J., Mason, C. E., Santoyo-Lopez, J., Setterquist, R. A., Shi, L., Shi, W., Smyth, G. K., Stralis-Pavese, N., Su, Z., Tong, W., Wang, C., Wang, J., Xu, J., Ye, Z., Yang, Y., Yu, Y., & Salit, M.
Venue
Nature Communications
Published
2014-09-25
Volume
5
Issue
1
Pages
5125-5125
ISSN
2041-1723 (ISSN-L)
OpenAlex
W2151876471
Cited by
124
References
13
FWCI
4.87
Type
journal article
Language
en
Open access
bronze
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures. https://app.cochat.ai/papers/10.1038/ncomms6125
The paper
Munro, S. A., Lund, S. P., Pine, P. S., Binder, H., Clevert, D.-A., Conesa, A., Dopazo, J., Fasold, M., Hochreiter, S., Hong, H., Jafari, N., Kreil, D. P., Łabaj, P. P., Li, S., Liao, Y., Lin, S. M., Meehan, J., Mason, C. E., Santoyo-Lopez, J., . . . Salit, M. (2014). Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures. Nature Communications. https://doi.org/10.1038/ncomms6125

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.