Systematic evaluation of spliced alignment programs for RNA-seq data
Open accessAbstract
Authors compare RNA-seq aligners on mouse and human data sets using benchmarks such as alignment yield, splice junction accuracy and suitability for transcript reconstruction. The work highlights the strength of each program and discusses outstanding needs in RNA-seq analysis. High-throughput RNA sequencing is an increasingly accessible method for studying gene structure and activity on a genome-wide scale. A critical step in RNA-seq data analysis is the alignment of partial transcript reads to a reference genome sequence. To assess the performance of current mapping software, we invited developers of RNA-seq aligners to process four large human and mouse RNA-seq data sets. In total, we compared 26 mapping protocols based on 11 programs and pipelines and found major performance differences between methods on numerous benchmarks, including alignment yield, basewise accuracy, mismatch and gap placement, exon junction discovery and suitability of alignments for transcript reconstruction. We observed concordant results on real and simulated RNA-seq data, confirming the relevance of the metrics employed. Future developments in RNA-seq alignment methods would benefit from improved placement of multimapped reads, balanced utilization of existing gene annotation and a reduced false discovery rate for splice junctions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Gene-finding tools such as Augustus can incorporate RNA-seq data to better annotate protein-coding transcripts, but perform worse on non-coding transcripts.
“Gene-finding tools such as Augustus [28] can incorporate RNA-seq data to better annotate protein-coding transcripts, but perform worse on non-coding transcripts [29].”
From A survey of best practices for RNA-seq data analysis (2016) · cites this paper as [29]
Accurate transcript reconstruction from short reads is difficult, and methods typically show substantial disagreement.
“In general, accurate transcript reconstruction from short reads is difficult, and methods typically show substantial disagreement [29].”
From A survey of best practices for RNA-seq data analysis (2016) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Sequence Alignment/Map format and SAMtools
Li, H., Handsaker, B., Wysoker, A., Fennell, T., Ruan, J., Homer, N., Marth, G., Abecasis, G., Durbin, R., & 1000 Genome Project Data Processing Subgroup (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp352 · Cited by 62,041 · see citing works
Fast gapped-read alignment with Bowtie 2
Langmead, B., & Salzberg, S. L. (2012) · Nature Methods · doi:10.1038/nmeth.1923 · Cited by 56,400 · see citing works
STAR: ultrafast universal RNA-seq aligner
Dobin, A., Davis, C. A., Schlesinger, F., Drenkow, J., Zaleski, C., Jha, S., Batut, P., Chaisson, M., & Gingeras, T. R. (2013) · Bioinformatics · doi:10.1093/bioinformatics/bts635 · Cited by 51,709 · see citing works
Trapnell, C., Williams, B. A., Pertea, G., Mortazavi, A., Kwan, G., van Baren, M. J., Salzberg, S. L., Wold, B. J., & Pachter, L. (2010) · Nature Biotechnology · doi:10.1038/nbt.1621 · Cited by 15,556 · see citing works
Kim, D., Pertea, G., Trapnell, C., Pimentel, H., Kelley, R., & Salzberg, S. L. (2013) · Genome Biology · doi:10.1186/gb-2013-14-4-r36 · Cited by 12,668 · see citing works
TopHat: discovering splice junctions with RNA-Seq
Trapnell, C., Pachter, L. S., & Salzberg, S. L. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp120 · Cited by 11,477 · see citing works
<tt>BLAT</tt> —The <tt>BLAST</tt> -Like Alignment Tool
Kent, W. J. (2002) · Genome Research · doi:10.1101/gr.229202 · Cited by 8,480 · see citing works
Landscape of transcription in human cells
Djebali, S., Davis, C. A., Merkel, A., Dobin, A., Lassmann, T., Mortazavi, A., Tanzer, A., Lagarde, J., Lin, W., Schlesinger, F., et al. (2012) · Nature · doi:10.1038/nature11233 · Cited by 5,187 · see citing works
Self-citationFinishing the euchromatic sequence of the human genome
International Human Genome Sequencing Consortium (2004) · Nature · doi:10.1038/nature03001 · Cited by 4,971 · see citing works
Fast and SNP-tolerant detection of complex variants and splicing in short reads
Wu, T. D., & Nacu, S. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq057 · Cited by 1,894 · see citing works
Wisdom of crowds for robust gene network inference
The DREAM5 Consortium, Marbach, D., Costello, J. C., Küffner, R., Vega, N. M., Prill, R. J., Camacho, D. M., Allison, K. R., Kellis, M., Collins, J. J., et al. (2012) · Nature Methods · doi:10.1038/nmeth.2016 · Cited by 1,667 · see citing works
Self-citationMouse genomic variation and its effect on phenotypes and gene regulation.
Keane, T. M., Goodstadt, L., Danecek, P., White, M. A., Wong, K., Yalcin, B., Heger, A., Agam, A., Slater, G., Goodson, M., et al. (2011) · Nature · doi:10.1038/nature10413 · Cited by 1,580 · see citing works
MapSplice: Accurate mapping of RNA-seq reads for splice junction discovery
Wang, K., Singh, D., Zeng, Z., Coleman, S. J., Huang, Y., Savich, G. L., He, X., Mieczkowski, P., Grimm, S. A., Perou, C. M., et al. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq622 · Cited by 1,193 · see citing works
Critical assessment of methods of protein structure prediction (CASP) — round x
Moult, J., Fidelis, K. A., Kryshtafovych, A., Schwede, T. F., & Tramontano, A. L. (2013) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.24452 · Cited by 673 · see citing works
The GEM mapper: fast, accurate and versatile alignment by filtration
Marco‐Sola, S., Sammeth, M., Guigó, R., & Ribeca, P. (2012) · Nature Methods · doi:10.1038/nmeth.2221 · Cited by 523 · see citing works
Self-citationAssemblathon 1: A competitive assessment of de novo short read assembly methodsPreprint
Earl, D. A., Bradnam, K. R., John, J. S., Darling, A. E., Lin, D., Fass, J., Yu, H. O. K., Buffalo, V., Zerbino, D. R., Diekhans, M., et al. (2011) · Genome Research · doi:10.1101/gr.126599.111 · Cited by 503 · see citing works
EGASP: the human ENCODE Genome Annotation Assessment Project.
Guigó, R., Flicek, P., Abril, J. F., Reymond, A., Lagarde, J., Denœud, F., Antonarakis, S. E., Ashburner, M., Bajić, V. B., Birney, E., et al. (2006) · Genome biology · doi:10.1186/gb-2006-7-s1-s2 · Cited by 341 · see citing works
Self-citationComparative analysis of RNA-Seq alignment algorithms and the RNA-Seq unified mapper (RUM)
Grant, G. R., Farkas, M. H., Pizarro, A., Lahens, N. F., Schug, J., Brunk, B. P., Stoeckert, C. J., Hogenesch, J. B., & Pierce, E. A. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr427 · Cited by 336 · see citing works
Self-citationTools for mapping high-throughput sequencing data
Fonseca, N. A., Rung, J., Brāzma, A., & Marioni, J. C. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts605 · Cited by 276 · see citing works
Critical assessment of methods of protein structure prediction (CASP)—round IX
Moult, J., Fidelis, K. A., Kryshtafovych, A., & Tramontano, A. L. (2011) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.23200 · Cited by 211 · see citing works
High levels of RNA-editing site conservation amongst 15 laboratory mouse strains
Danecek, P., Nellåker, C., McIntyre, R. E., Buendia-Buendia, J. E., Bumpstead, S., Ponting, C. P., Jonathan Flint, C., Durbin, R. M., Keane, T., & Adams, D. J. (2012) · Genome biology · doi:10.1186/gb-2012-13-4-r26 · Cited by 160 · see citing works
PASS: a program to align short sequences
Campagna, D., Albiero, A., Bilardi, A., Caniato, E., Forcato, C., Manavski, S. A., Vitulo, N., & Valle, G. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp087 · Cited by 122 · see citing works
RNA‐Seq Read Alignments with PALMapper
Jean, G., Kahles, A., Sreedharan, V. T., Bona, F. D., & Rätsch, G. (2010) · Current Protocols in Bioinformatics · doi:10.1002/0471250953.bi1106s32 · Cited by 59 · see citing works
Self-citationAll Your Base: a fast and accurate probabilistic approach to base calling
Massingham, T., & Goldman, N. (2012) · Genome biology · doi:10.1186/gb-2012-13-2-r13 · Cited by 49 · see citing works
Self-citationComparative analysis of information contents relevant to recognition of introns in many species
Iwata, H., & Gotoh, O. (2011) · BMC Genomics · doi:10.1186/1471-2164-12-45 · Cited by 41 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A survey of best practices for RNA-seq data analysis
Conesa, A., Madrigal, P., Tarazona, S., Gomez-Cabrero, D., Cervera, A., McPherson, A., Szcześniak, M. W., Gaffney, D. J., Elo, L. L., Zhang, X., et al. (2016) · Genome Biology · doi:10.1186/s13059-016-0881-8 · Cited by 2,794 · see citing works
Kukurba, K. R., & Montgomery, S. B. (2015) · Cold Spring Harbor Protocols · doi:10.1101/pdb.top084970 · Cited by 924 · see citing works
Specification of tissue-resident macrophages during organogenesis
Mass, E., Ballesteros, I., Farlik, M., Halbritter, F., Günther, P., Crozet, L., Jacome-Galarza, C. E., Händler, K., Klughammer, J., Kobayashi, Y., et al. (2016) · Science · doi:10.1126/science.aaf4238 · Cited by 891 · see citing works
Circular RNAs: analysis, expression and potential functions
Barrett, S. P., & Salzman, J. (2016) · Development · doi:10.1242/dev.128074 · Cited by 859 · see citing works
Assessment of transcript reconstruction methods for RNA-seq
The RGASP Consortium, Steijger, T., Abril, J. F., Engström, P. G., Kokocinski, F., Hubbard, T. J., Guigó, R., Harrow, J., & Bertone, P. (2013) · Nature Methods · doi:10.1038/nmeth.2714 · Cited by 699 · see citing works
Self-citationTitin mutations in iPS cells define sarcomere insufficiency as a cause of dilated cardiomyopathy
Hinson, J. T., Chopra, A., Nafissi, N., Polacheck, W. J., Benson, C. C., Swist, S., Gorham, J., Yang, L., Schafer, S., Sheng, C. C., et al. (2015) · Science · doi:10.1126/science.aaa5458 · Cited by 638 · see citing works
Tsuchida, T., Lee, Y. A., Fujiwara, N., Ybanez, M., Allen, B., Martins, S., Fiel, M. I., Goossens, N., Chou, H.-I., Hoshida, Y., et al. (2018) · Journal of Hepatology · doi:10.1016/j.jhep.2018.03.011 · Cited by 583 · see citing works
Early prediction of circulatory failure in the intensive care unit using machine learning
Hyland, S. L., Faltys, M., Hüser, M., Lyu, X., Gumbsch, T., Esteban, C., Bock, C., Horn, M., Moor, M., Rieck, B., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0789-4 · Cited by 477 · see citing works
Self-citationSahraeian, S. M. E., Mohiyuddin, M., Sebra, R., Tilgner, H., Afshar, P. T., Au, K. F., Bani Asadi, N., Gerstein, M. B., Wong, W. H., Snyder, M. P., et al. (2017) · Nature Communications · doi:10.1038/s41467-017-00050-4 · Cited by 376 · see citing works
Zhao, S., & Zhang, B. (2015) · BMC Genomics · doi:10.1186/s12864-015-1308-8 · Cited by 130 · see citing works
Anvar, S. Y., Allard, G., Tseng, E., Sheynkman, G. M., de Klerk, E., Vermaat, M., Yin, R. H., Johansson, H. E., Ariyurek, Y., den Dunnen, J. T., et al. (2018) · Genome Biology · doi:10.1186/s13059-018-1418-0 · Cited by 118 · see citing works
From tissues to cell types and back: single-cell gene expression analysis of tissue architecture
Chen, X., Teichmann, S. A., & Meyer, K. B. (2018) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-080917-013452 · Cited by 112 · see citing works
Hayer, K. E., Pizarro, A., Lahens, N. F., Hogenesch, J. B., & Grant, G. R. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv488 · Cited by 93 · see citing works
Self-citationAnalytical tools and current challenges in the modern era of neuroepigenomics
Maze, I., Shen, L., Zhang, B., Garcia, B. A., Shao, N., Mitchell, A., Sun, H., Akbarian, S., Allis, C. D., & Nestler, E. J. (2014) · Nature Neuroscience · doi:10.1038/nn.3816 · Cited by 100 · see citing works
Decoding neuroproteomics: integrating the genome, translatome and functional anatomy
Kitchen, R. R., Rozowsky, J. S., Gerstein, M. B., & Nairn, A. C. (2014) · Nature Neuroscience · doi:10.1038/nn.3829 · Cited by 63 · see citing works
- Authors
- The RGASP Consortium, Engström, P. G., Steijger, T., Sipos, B., Grant, G. R., Kahles, A., Rätsch, G., Goldman, N., Hubbard, T. J., Harrow, J., Guigó, R., & Bertone, P.
- Affiliations
- European Bioinformatics Institute (GB)
- University of Pennsylvania (US)
- Max Planck Society (DE)
- Wellcome Sanger Institute (GB)
- Universitat Pompeu Fabra (ES)
- ORCID
- Venue
- Nature Methods
- Published
- 2013-11-03
- Volume
- 10
- Issue
- 12
- Pages
- 1185-1191
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- PMC
- PMC4018468
- OpenAlex
- W2054576015
- Cited by
- 513
- References
- 25
- Citations by year
- 2026: 13 · 2025: 17 · 2024: 29 · 2023: 23 · 2022: 32 · 2021: 42 · 2020: 31 · 2019: 47 · 2018: 52 · 2017: 68 · 2016: 66 · 2015: 56 · 2014: 32 · 2013: 4 · 2012: 1
- FWCI
- 23.57
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-sa
- www.springer.com/tdm (unspecified)
- Locations
- Nature Methods (publishedVersion, cc-by-nc-sa)
- PubMed Central (publishedVersion, cc-by-nc-sa)
- PubMed (publishedVersion)
- Research Portal (King's College London) (publishedVersion, other-oa)
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (publishedVersion, other-oa)
- Cold Spring Harbor Laboratory Institutional Repository (Cold Spring Harbor Laboratory) (acceptedVersion)
- Europe PMC (PubMed Central) (acceptedVersion, cc-by-nc-sa)
- HAL (submittedVersion)
- Repositori digital de la UPF (Universitat Pompeu Fabra) (submittedVersion, cc-by-nc-sa)
- Apollo (University of Cambridge) (submittedVersion, cc-by-nc-sa)
- Padua Research Archive (University of Padova) (submittedVersion, cc-by-nc-sa)
- Apollo
- Funders
- Wellcome Trust (26205, WT09805, 098051)
- European Molecular Biology Laboratory
- National Institutes of Health (5u54hg004555-04)
- National Human Genome Research Institute (U54-HG004555, R01HG006272, U41HG007234, U54HG004557)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA Research and Splicing
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Keywords
- RNA-seq alignment, spliced alignment, false discovery rate, genome annotation
- MeSH terms
- Animals, Chromosome Mapping, Exons, False Positive Reactions, Humans, RNA Splicing, RNA, Messenger, Software, Reproducibility of Results, Sequence Alignment, Sequence Analysis, RNA, Computational Biology, K562 Cells, Mice, High-Throughput Nucleotide Sequencing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
