CoChat

Characterization and improvement of RNA-Seq precision in quantitative transcript expression profiling

Open access

The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.

Abstract

Motivation: Measurement precision determines the power of any analysis to reliably identify significant signals, such as in screens for differential expression, independent of whether the experimental design incorporates replicates or not. With the compilation of large-scale RNA-Seq datasets with technical replicate samples, however, we can now, for the first time, perform a systematic analysis of the precision of expression level estimates from massively parallel sequencing technology. This then allows considerations for its improvement by computational or experimental means. Results: We report on a comprehensive study of target identification and measurement precision, including their dependence on transcript expression levels, read depth and other parameters. In particular, an impressive recall of 84% of the estimated true transcript population could be achieved with 331 million 50 bp reads, with diminishing returns from longer read lengths and even less gains from increased sequencing depths. Most of the measurement power (75%) is spent on only 7% of the known transcriptome, however, making less strongly expressed transcripts harder to measure. Consequently, <30% of all transcripts could be quantified reliably with a relative error <20%. Based on established tools, we then introduce a new approach for mapping and analysing sequencing reads that yields substantially improved performance in gene expression profiling, increasing the number of transcripts that can reliably be quantified to over 40%. Extrapolations to higher sequencing depths highlight the need for efficient complementary steps. In discussion we outline possible experimental and computational strategies for further improvements in quantification precision. Contact: rnaseq10@boku.ac.at Supplementary information: Supplementary data are available at Bioinformatics online.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/bioinformatics/btr247

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1. Statistics of identified and reliably measured transcripts. The plot compares, for different read processing methods, the number of transcripts identified (white and light bars), as well as the number of transcripts that could be measured reliably, i.e. where expression levels could be quantified with an error of 20% or less (black and dark bars). The grey bars are for gene models constructed de novo. The black and white bars are for known spliceforms as given by the EnsEMBL gene models. The alternate y-axis on the right gives the corresponding fraction of all spliceforms known (EnsEMBL, numbers given in brackets). The last table row gives the ratio of reliably measured transcript relative to the number of identified transcripts. The plot considers four different approaches of computing transcript expression from the observed reads. The first couple of bars assesses read alignments to the transcriptome from Bowtie with subsequent calculation of expression levels from the unambiguously mapping reads. Only 24 081 spliceforms could be measured reliably. The next group shows results for the established programs TopHat and Cufflinks, where an alignment of reads to the genome is followed by expression level estimates from de novo constructed gene models. While more transcripts can now be assessed reliably (35 405), also an extremely large number of spliceforms is predicted (off-scale). When Cufflinks is allowed to use the known EnsEMBL gene models, the number of reliably measurable transcripts increases to 39 116 or 28% of all known spliceforms. The approach of read alignment to the transcriptome by Bowtie combined with gene model based expression level estimates by Cufflinks identifies the largest number of known transcripts (72%) and allows the reliable measurement of 56 980, that is 41% of all known transcripts.
Fig. 1. Statistics of identified and reliably measured transcripts. The plot compares, for different read processing methods, the number of transcripts identified (white and light bars), as well as the number of transcripts that could be measured reliably, i.e. where expression levels could be quantified with an error of 20% or less (black and dark bars). The grey bars are for gene models constructed de novo. The black and white bars are for known spliceforms as given by the EnsEMBL gene models. The alternate y-axis on the right gives the corresponding fraction of all spliceforms known (EnsEMBL, numbers given in brackets). The last table row gives the ratio of reliably measured transcript relative to the number of identified transcripts. The plot considers four different approaches of computing transcript expression from the observed reads. The first couple of bars assesses read alignments to the transcriptome from Bowtie with subsequent calculation of expression levels from the unambiguously mapping reads. Only 24 081 spliceforms could be measured reliably. The next group shows results for the established programs TopHat and Cufflinks, where an alignment of reads to the genome is followed by expression level estimates from de novo constructed gene models. While more transcripts can now be assessed reliably (35 405), also an extremely large number of spliceforms is predicted (off-scale). When Cufflinks is allowed to use the known EnsEMBL gene models, the number of reliably measurable transcripts increases to 39 116 or 28% of all known spliceforms. The approach of read alignment to the transcriptome by Bowtie combined with gene model based expression level estimates by Cufflinks identifies the largest number of known transcripts (72%) and allows the reliable measurement of 56 980, that is 41% of all known transcripts.CC BY-NC · PubMed Central
Fig. 2. Standard deviation versus expression level. The plot shows the variation across three technical replicate measurements (standard deviation, y-axis), with each discernible dot representing a transcript target. In shaded areas, the grey level represents density, with dark shading indicating higher densities. The standard deviation is in general larger for transcripts with lower mean expression level (x-axis). More strongly expressed transcripts could often be measured reliably, with a relative error of 20% or less. Interestingly, just 41% of all transcript targets could be measured that precisely (below the horizontal dashed line). Of the 41% most strongly expressed transcripts (to the right of the vertical dashed line), on the other hand, 84% could be measured reliably (below the horizontal dashed line). This is reflected by the high density of targets on the right (dark shading) falling largely below the horizontal line, which is not the case to the left of the vertical dashed line.
Fig. 2. Standard deviation versus expression level. The plot shows the variation across three technical replicate measurements (standard deviation, y-axis), with each discernible dot representing a transcript target. In shaded areas, the grey level represents density, with dark shading indicating higher densities. The standard deviation is in general larger for transcripts with lower mean expression level (x-axis). More strongly expressed transcripts could often be measured reliably, with a relative error of 20% or less. Interestingly, just 41% of all transcript targets could be measured that precisely (below the horizontal dashed line). Of the 41% most strongly expressed transcripts (to the right of the vertical dashed line), on the other hand, 84% could be measured reliably (below the horizontal dashed line). This is reflected by the high density of targets on the right (dark shading) falling largely below the horizontal line, which is not the case to the left of the vertical dashed line.CC BY-NC · PubMed Central
Fig. 3. Cumulative distribution of read alignments across transcript targets. The fraction of read alignments is plotted (y-axis) that has been mapped to a certain percentage of transcript targets (x-axis). Over 75% of all read alignments cover less than 7% of the known transcriptome (circle symbol). Two particular positions are marked by vertical lines in the figure: The 41% of targets with the highest expression are to the left of the first line (dotted). The vast majority of read alignments (99.5%) has been assigned to these targets, supporting a reliable measurement of their expression levels. Consequently, most of them (84%) could be determined with an error of 20% or less. On average, 67% of all transcript targets were identified in a measurement and this is marked by the second line (dashed). A substantial number of transcript targets falls between the two lines, receiving as few as only one read alignment. Consequently, most of these targets could not be quantified reliably. The remaining 33% of transcript targets falling to the right of the second line (dashed) were either undetected or not expressed.
Fig. 3. Cumulative distribution of read alignments across transcript targets. The fraction of read alignments is plotted (y-axis) that has been mapped to a certain percentage of transcript targets (x-axis). Over 75% of all read alignments cover less than 7% of the known transcriptome (circle symbol). Two particular positions are marked by vertical lines in the figure: The 41% of targets with the highest expression are to the left of the first line (dotted). The vast majority of read alignments (99.5%) has been assigned to these targets, supporting a reliable measurement of their expression levels. Consequently, most of them (84%) could be determined with an error of 20% or less. On average, 67% of all transcript targets were identified in a measurement and this is marked by the second line (dashed). A substantial number of transcript targets falls between the two lines, receiving as few as only one read alignment. Consequently, most of these targets could not be quantified reliably. The remaining 33% of transcript targets falling to the right of the second line (dashed) were either undetected or not expressed.CC BY-NC · PubMed Central
Fig. 4. Transcripts with reliable quantification versus read depth. This graph plots the number of of transcript targets that could be measured reliably with a relative error of 20% or less versus the number of read alignments (x-axis). The total number of generated reads is given in parentheses below. Additional tick marks indicate proportions of a flowcell or the number of flowcells worth of sequencing. The alternate y-axis on the right shows the percentage of all known transcripts measured reliably. The solid line shows the results for the introduced combined quantification approach (Bowtie+Cufflinks+model). The dependency of the number of transcripts with reliable quantification on the number of read alignments can be described as a function with a sigmoid shape (regression P<10−15). The circle symbol indicates 41%, as achieved with an entire flowcell per replicate (331 million reads). Extrapolation of the fitted sigmoid suggests that about 60% can be reached at 10 billion reads, highlighting the need for efficient complementary steps. See text for discussion. See text for discussion. In comparison, the plus sign shows the corresponding result for an established standard approach (TopHat+Cufflinks+model), 28% of all known transcripts. The data shown is for the total, pooled set of reads.
Fig. 4. Transcripts with reliable quantification versus read depth. This graph plots the number of of transcript targets that could be measured reliably with a relative error of 20% or less versus the number of read alignments (x-axis). The total number of generated reads is given in parentheses below. Additional tick marks indicate proportions of a flowcell or the number of flowcells worth of sequencing. The alternate y-axis on the right shows the percentage of all known transcripts measured reliably. The solid line shows the results for the introduced combined quantification approach (Bowtie+Cufflinks+model). The dependency of the number of transcripts with reliable quantification on the number of read alignments can be described as a function with a sigmoid shape (regression P<10−15). The circle symbol indicates 41%, as achieved with an entire flowcell per replicate (331 million reads). Extrapolation of the fitted sigmoid suggests that about 60% can be reached at 10 billion reads, highlighting the need for efficient complementary steps. See text for discussion. See text for discussion. In comparison, the plus sign shows the corresponding result for an established standard approach (TopHat+Cufflinks+model), 28% of all known transcripts. The data shown is for the total, pooled set of reads.CC BY-NC · PubMed Central
Fig. 5. Transcript identification versus read depth. This plot shows the number of detected transcript targets versus the number of read alignments (x-axis). The total number of generated reads is given in parentheses below. Additional tick marks indicate proportions of a flowcell or the number of flowcells worth of sequencing. The alternate y-axis on the right shows the percentage of all known transcripts detected. The solid line shows the results for the introduced combined quantification approach (Bowtie+Cufflinks+model). The dependency of the number of transcripts identified on the number of read alignments can be described as a function with a sigmoid shape (regression P<10−12). The circle symbol indicates 72%, as obtained with the entire set of 993 million reads. The remaining 28% were either undetected or not expressed in the studied sample. Extrapolation of the sigmoid fit suggests that even with an infinite number of reads only marginally more transcripts would be expected to be identified, yielding an estimate of 20% of transcript targets that are actually not expressed. As a consequence, this experiment reached a target recall of 90% of the estimated true transcript population of the sample. A single flowcell already achieved 84%. In comparison, the plus sign plots the corresponding result for an established standard approach (TopHat+Cufflinks+model), which identified 63% of all known transcripts, which is 79% of the estimated true transcript population. Results are shown for the entire dataset, pooling reads from all replicates.
Fig. 5. Transcript identification versus read depth. This plot shows the number of detected transcript targets versus the number of read alignments (x-axis). The total number of generated reads is given in parentheses below. Additional tick marks indicate proportions of a flowcell or the number of flowcells worth of sequencing. The alternate y-axis on the right shows the percentage of all known transcripts detected. The solid line shows the results for the introduced combined quantification approach (Bowtie+Cufflinks+model). The dependency of the number of transcripts identified on the number of read alignments can be described as a function with a sigmoid shape (regression P<10−12). The circle symbol indicates 72%, as obtained with the entire set of 993 million reads. The remaining 28% were either undetected or not expressed in the studied sample. Extrapolation of the sigmoid fit suggests that even with an infinite number of reads only marginally more transcripts would be expected to be identified, yielding an estimate of 20% of transcript targets that are actually not expressed. As a consequence, this experiment reached a target recall of 90% of the estimated true transcript population of the sample. A single flowcell already achieved 84%. In comparison, the plus sign plots the corresponding result for an established standard approach (TopHat+Cufflinks+model), which identified 63% of all known transcripts, which is 79% of the estimated true transcript population. Results are shown for the entire dataset, pooling reads from all replicates.CC BY-NC · PubMed Central
Fig. 6. Comparison of measurement variation. The graph compares the rescaled cumulative distributions of the standard deviation for alternative technologies and data processing protocols.
Fig. 6. Comparison of measurement variation. The graph compares the rescaled cumulative distributions of the standard deviation for alternative technologies and data processing protocols.CC BY-NC · PubMed Central
Fig. 7. Alternative RNA-Seq application schemas. (a) In an iterative approach, high-abundance transcripts can be identified in low-read sequencing runs, followed by iterative subtraction of the sequences dominating each sample. A profile from the combined runs promises higher measurement precision of expression levels for weakly to moderately expressed transcripts. (b) After normalization of an aliquot (top row), the strength of RNA-Seq in de novo sequence discovery can be exploited for the compilation of a comprehensive target library, against which a custom microarray can then be designed easily (Leparc et al., 2009) The remaining aliquot can then be quantitatively profiled on this optimized array (bottom row). The performance of both approaches of course depends on the quality of the subtraction or normalization step, respectively.
Fig. 7. Alternative RNA-Seq application schemas. (a) In an iterative approach, high-abundance transcripts can be identified in low-read sequencing runs, followed by iterative subtraction of the sequences dominating each sample. A profile from the combined runs promises higher measurement precision of expression levels for weakly to moderately expressed transcripts. (b) After normalization of an aliquot (top row), the strength of RNA-Seq in de novo sequence discovery can be exploited for the compilation of a comprehensive target library, against which a custom microarray can then be designed easily (Leparc et al., 2009) The remaining aliquot can then be quantitatively profiled on this optimized array (bottom row). The performance of both approaches of course depends on the quality of the subtraction or normalization step, respectively.CC BY-NC · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 42 references

Research works printed but not linked: 2

  1. Bolstad, 2004, Low level analysis of high-density oligonucleotide array data: background, normalization and summarization, PhD Thesis
  2. Thas, 2010, Probabilistic allelic read calling: a quasi-Poisson mixed model for the analysis of allelic read counts

Cited by

See all 167

Metadata

Authors
Łabaj, P. P., Leparc, G. G., Linggi, B. E., Markillie, L. M., Wiley, H. S., & Kreil, D. P.
Affiliations
Pacific Northwest National Laboratory (US)
Corresponding author
Kreil, David P.
Venue
Bioinformatics
Published
2011-06-14
Volume
27
Issue
13
Pages
i383-i391
ISSN
  • 1367-4803 (ISSN-L)
  • 1367-4811
OpenAlex
W2163871842
Cited by
141
References
42
Citations by year
2026: 1 · 2025: 2 · 2024: 7 · 2023: 2 · 2022: 5 · 2021: 7 · 2020: 3 · 2019: 9 · 2018: 7 · 2017: 8 · 2016: 14 · 2015: 15 · 2014: 17 · 2013: 23 · 2012: 17
FWCI
7.91
Type
journal article
Language
en
Open access
Locations
  • Bioinformatics (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • HighWire Press Open Archive (submittedVersion)
Funders
  • U.S. Department of Energy
  • Vienna Science and Technology Fund
  • Austrian Institute of Technology
  • Biological and Environmental Research
  • Laboratory Directed Research and Development
  • Pacific Northwest National Laboratory
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
Keywords
RNA-seq, measurement precision, transcript quantification, sequencing depth, read length, technical replicates, transcriptome coverage, differential gene expression, read mapping, gene expression profiling
MeSH terms
Cell Line, Humans, RNA, Software, Sequence Analysis, RNA, Gene Expression Profiling, Microarray Analysis, High-Throughput Nucleotide Sequencing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Characterization and improvement of RNA-Seq precision in quantitative transcript expression profiling. https://app.cochat.ai/papers/10.1093/bioinformatics/btr247
The paper
Łabaj, P. P., Leparc, G. G., Linggi, B. E., Markillie, L. M., Wiley, H. S., & Kreil, D. P. (2011). Characterization and improvement of RNA-Seq precision in quantitative transcript expression profiling. Bioinformatics. https://doi.org/10.1093/bioinformatics/btr247

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.