Benchmark analysis of algorithms for determining and quantifying full-length mRNA splice forms from RNA-seq data
Open accessPublished version of a preprint
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Motivation: Because of the advantages of RNA sequencing (RNA-Seq) over microarrays, it is gaining widespread popularity for highly parallel gene expression analysis. For example, RNA-Seq is expected to be able to provide accurate identification and quantification of full-length splice forms. A number of informatics packages have been developed for this purpose, but short reads make it a difficult problem in principle. Sequencing error and polymorphisms add further complications. It has become necessary to perform studies to determine which algorithms perform best and which if any algorithms perform adequately. However, there is a dearth of independent and unbiased benchmarking studies. Here we take an approach using both simulated and experimental benchmark data to evaluate their accuracy. Results: We conclude that most methods are inaccurate even using idealized data, and that no method is highly accurate once multiple splice forms, polymorphisms, intron signal, sequencing errors, alignment errors, annotation errors and other complicating factors are present. These results point to the pressing need for further algorithm development. Availability and implementation: Simulated datasets and other supporting information can be found at http://bioinf.itmat.upenn.edu/BEERS/bp2 Supplementary information: Supplementary data are available at Bioinformatics online. Contact: hayer@upenn.edu
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Even so, it is common to observe extreme variability in the coverage of RNA-seq fragments along transcripts that is purely technical and sample-specific17 and not explained by current bias models, which confounds current methods designed to identify and quantify transcripts18”
From Modeling of RNA-seq fragment sequence bias reduces systematic errors in transcript abundance estimation (2016) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STAR: ultrafast universal RNA-seq aligner
Dobin, A., Davis, C. A., Schlesinger, F., Drenkow, J., Zaleski, C., Jha, S., Batut, P., Chaisson, M., & Gingeras, T. R. (2013) · Bioinformatics · doi:10.1093/bioinformatics/bts635 · Cited by 51,709 · cited 1×
“For alignment-guided methods, we aligned the reads with TopHat2 [20] (hereafter referred to as TopHat) and STAR [21].”
RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome
Li, B., & Dewey, C. N. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-323 · Cited by 22,179 · cited 2×
“, 2015), RSEM (Li and Dewey, 2011) and IsoEM (Nicolae et al.”
BackgroundHTSeq—a Python framework to work with high-throughput sequencing data
Anders, S., Pyl, P. T., & Huber, W. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btu638 · Cited by 21,464 · cited 1×
“These results point to the need for new algorithm development to solve this difficult, but important, problem.”
BackgroundFull-length transcriptome assembly from RNA-Seq data without a reference genome
Grabherr, M. G., Haas, B. J., Yassour, M., Levin, J. Z., Thompson, D. A., Amit, I., Adiconis, X., Fan, L., Raychowdhury, R., Zeng, Q., et al. (2011) · Nature Biotechnology · doi:10.1038/nbt.1883 · Cited by 21,039
Trapnell, C., Williams, B. A., Pertea, G., Mortazavi, A., Kwan, G., van Baren, M. J., Salzberg, S. L., Wold, B. J., & Pachter, L. (2010) · Nature Biotechnology · doi:10.1038/nbt.1621 · Cited by 15,556
StringTie enables improved reconstruction of a transcriptome from RNA-seq reads
Pertea, M., Pertea, G. M., Antonescu, C. M., Chang, T.-C., Mendell, J. T., & Salzberg, S. L. (2015) · Nature Biotechnology · doi:10.1038/nbt.3122 · Cited by 14,298 · cited 1×
“They assessed performance by correlation; however that does not account for the accuracy of absolute intensity levels, just how the values co-vary.”
BackgroundKim, D., Pertea, G., Trapnell, C., Pimentel, H., Kelley, R., & Salzberg, S. L. (2013) · Genome Biology · doi:10.1186/gb-2013-14-4-r36 · Cited by 12,668 · cited 1×
“For alignment-guided methods, we aligned the reads with TopHat2 [20] (hereafter referred to as TopHat) and STAR [21].”
MethodThe Human Genome Browser at UCSC
Kent, W. J., Sugnet, C. W., Furey, T. S., Roskin, K. M., Pringle, T. H., Zahler, A. M., & Haussler, A. D. (2002) · Genome Research · doi:10.1101/gr.229102 · Cited by 10,400 · cited 1×
“BEERS starts with a genome and a set of gene annotations; here we used mouse genome build mm9 [9] and RefSeq genes [10], as aligned to the reference genome by the UCSC genome browser [11].”
MethodInitial sequencing and comparative analysis of the mouse genome
Mouse Genome Sequencing Consortium (2002) · Nature · doi:10.1038/nature01262 · Cited by 7,187
Differential analysis of gene regulation at transcript resolution with RNA-seq
Trapnell, C., Hendrickson, D. G., Sauvageau, M., Goff, L., Rinn, J. L., & Pachter, L. (2013) · Nature Biotechnology · doi:10.1038/nbt.2450 · Cited by 3,492 · cited 1×
“These results point to the need for new algorithm development to solve this difficult, but important, problem.”
Background
Cited by
Detecting circular RNAs: bioinformatic and experimental challenges
Szabo, L., & Salzman, J. (2016) · Nature Reviews Genetics · doi:10.1038/nrg.2016.114 · Cited by 683
“However, as has already been noted with respect to linear splice detection, increased coverage improves sensitivity but actually results in a higher false-positive rate 36,53,56 , and longer reads do not sufficiently address this issue 56 .”
BackgroundResultPetryszak, R., Keays, M., Tang, Y. A., Fonseca, N. A., Barrera, E., Burdett, T., Füllgrabe, A., Fuentes, A. M.-P., Jupp, S., Koskinen, S., et al. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1045 · Cited by 530
Modular, efficient and constant-memory single-cell RNA-seq preprocessing
Melsted, P., Booeshaghi, A. S., Liu, L., Gao, F., Lu, L., Min, K. H., da Veiga Beltrame, E., Hjörleifsson, K. E., Gehring, J., & Pachter, L. (2021) · Nature Biotechnology · doi:10.1038/s41587-021-00870-2 · Cited by 510
Sahraeian, S. M. E., Mohiyuddin, M., Sebra, R., Tilgner, H., Afshar, P. T., Au, K. F., Bani Asadi, N., Gerstein, M. B., Wong, W. H., Snyder, M. P., et al. (2017) · Nature Communications · doi:10.1038/s41467-017-00050-4 · Cited by 376
“On MCF7-100 and MCF7-300 samples, Cuf fl inks-TopHat prediction sets were not enriched in any MCF7 or breast cancer-related gene expression study, while StringTie-HISAT2 ’ s and Salmon-SMEM ’ s top over-expressed genes were highly enriched in many MCF7 and breast cancer cell line-related gene sets (Supplementary Data 1…”
BackgroundMethodInfluentialUnderstanding sequencing data as compositions: an outlook and review
Quinn, T. P., Erb, I., Richardson, M. F., & Crowley, T. M. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty175 · Cited by 350
“, [17]; [54]; [63]; [26]; [35]; [59]; [61]; [65]; [49]).”
Alternative Splicing May Not Be the Key to Proteome Complexity
Tress, M. L., Abascal, F., & Valencia, A. (2017) · Trends in Biochemical Sciences · doi:10.1016/j.tibs.2016.08.008 · Cited by 339
Rapid and Dynamic Alternative Splicing Impacts the Arabidopsis Cold Response Transcriptome
Calixto, C. P. G., Guo, W., James, A. B., Tzioutziou, N. A., Entizne, J. C., Panter, P. E., Knight, H., Nimmo, H. G., Zhang, R., & Brown, J. W. S. (2018) · The Plant Cell · doi:10.1105/tpc.18.00177 · Cited by 298
“In addition, accuracy also decreases with increasing numbers of isoforms in a gene (Hayer et al., 2015).”
BackgroundZhang, R., Calixto, C. P. G., Marquez, Y., Venhuizen, P., Tzioutziou, N. A., Guo, W., Spensley, M., Entizne, J. C., Lewandowska, D., ten Have, S., et al. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkx267 · Cited by 290
Simulation-based comprehensive benchmarking of RNA-seq aligners
Baruzzo, G., Hayer, K. E., Kim, E. J., Di Camillo, B., FitzGerald, G. A., & Grant, G. R. (2017) · Nature Methods · doi:10.1038/nmeth.4106 · Cited by 277
Self-citationRao, M. S., Van Vleet, T. R., Ciurlionis, R., Buck, W. R., Mittelstadt, S. W., Blomme, E. A. G., & Liguori, M. J. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2018.00636 · Cited by 262
Accurate assembly of transcripts through phase-preserving graph decomposition
Shao, M., & Kingsford, C. (2017) · Nature Biotechnology · doi:10.1038/nbt.4020 · Cited by 217
“The accuracies of existing methods are still very low, especially for lowly expressed transcripts and those genes with multiple spliced isoforms [15].”
Next-generation biology: Sequencing and data analysis approaches for non-model organisms
da Fonseca, R. R., Albrechtsen, A., Themudo, G. E., Ramos-Madrigal, J., Sibbesen, J. A., Maretty, L., Zepeda-Mendoza, M. L., Campos, P. F., Heller, R., & Pereira, R. J. (2016) · Marine Genomics · doi:10.1016/j.margen.2016.04.012 · Cited by 208
Love, M. I., Hogenesch, J. B., & Irizarry, R. A. (2016) · Nature Biotechnology · doi:10.1038/nbt.3682 · Cited by 186
Self-citationGene-level differential analysis at transcript-level resolution
Yi, L., Pimentel, H., Bray, N., & Pachter, L. S. (2018) · Genome biology · doi:10.1186/s13059-018-1419-z · Cited by 179
“While non-uniformity in coverage is prevalent in RNA-seq [19], it is particularly extreme in variants of RNA-seq that enrich for 5′ or 3′ sequences.”
BackgroundMost Alternative Isoforms Are Not Functionally Important
Tress, M. L., Abascal, F., & Valencia, A. (2017) · Trends in Biochemical Sciences · doi:10.1016/j.tibs.2017.04.002 · Cited by 94
Liu, R., Loraine, A. E., & Dickerson, J. A. (2014) · BMC Bioinformatics · doi:10.1186/s12859-014-0364-4 · Cited by 106
“Another two publications [18,19] benchmark methods and algorithms for transcript reconstruction and quantification.”
Nazarov, P. V., Muller, A., Kaoma, T., Nicot, N., Máximo, C., Birembaut, P. L., Tran, N. L., Dittmar, G., & Vallar, L. (2017) · BMC Genomics · doi:10.1186/s12864-017-3819-y · Cited by 99
“This observation supports the statement that analysis of differential exon usage based only on RNA-seq data may be inadvisable [4, 5].”
ResultAn analysis of tissue-specific alternative splicing at the protein level
Rodríguez, J. M., Pozo, F., Domenico, T. D., Vázquez, J., & Tress, M. L. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008287 · Cited by 99
LIQA: long-read isoform quantification and analysis
Hu, Y., Li, F., Chen, X., Zhong, J. F., Li, M., & Wang, K. (2021) · Genome biology · doi:10.1186/s13059-021-02399-8 · Cited by 88
What to compare and how: Comparative transcriptomics for Evo‐Devo
Roux, J., Rosikiewicz, M., & Robinson‐Rechavi, M. (2015) · Journal of Experimental Zoology Part B Molecular and Developmental Evolution · doi:10.1002/jez.b.22618 · Cited by 86
“Although RNA-seq facilitates the reconstruction of the pool of all isoforms for each gene, which then could ideally be examined in isolation, this task still proves to be very challenging (Hayer et al., 2014).”
BackgroundRapazote-Flores, P., Bayer, M., Milne, L., Mayer, C., Fuller, J., Guo, W., Hedley, P. E., Morris, J., Halpin, C., Kam, J., et al. (2019) · BMC Genomics · doi:10.1186/s12864-019-6243-7 · Cited by 73
“Such assemblies are typically inaccurate because first, they generate a large proportion of mis-assembled transcripts and second, they fail to assemble thousands of real transcripts present in the sample dataset (Pertea et al., 2015; Hayer et al., 2015).”
BackgroundQuantitative single-cell transcriptomics
Ziegenhain, C., Vieth, B., Parekh, S., Hellmann, I., & Enard, W. (2018) · Briefings in Functional Genomics · doi:10.1093/bfgp/ely009 · Cited by 61
“Several algorithms have been developed to quantify transcripts [102, 105, 113], but a correct transcript-level quantification is challenging for full-length bulk RNA-seq [95, 114].”
Contributions of alternative splicing to muscle type development and function
Nikonova, E., Kao, S.-Y., & Spletter, M. L. (2020) · Seminars in Cell & Developmental Biology · doi:10.1016/j.semcdb.2020.02.003 · Cited by 59
ORF Capture-Seq as a versatile method for targeted identification of full-length isoforms
Sheynkman, G., Tuttle, K. S., Laval, F., Tseng, E., Underwood, J. G., Yu, L., Dong, D., Smith, M. L., Sebra, R., Willems, L., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-16174-z · Cited by 56
Bridging the gap between reference and real transcriptomes
Morillon, A., & Gautheret, D. (2019) · Genome biology · doi:10.1186/s13059-019-1710-7 · Cited by 60
“As said by Hayer et al. [78], “ with more reads, most algorithms find more ways to go wrong ” .”
BackgroundLiu, Q., Hu, Y., Stucky, A., Li, F., Zhong, J. F., & Wang, K. (2020) · BMC Genomics · doi:10.1186/s12864-020-07207-4 · Cited by 71
“Also, the qRT-PCR measurements of gene/isoform expressions from this sample were used to benchmark and optimize computational tools [57 – 61].”
MethodChallenges and best practices in omics benchmarking
Brooks, T. G., Lahens, N. F., Mrčela, A., & Grant, G. R. (2024) · Nature Reviews Genetics · doi:10.1038/s41576-023-00679-6 · Cited by 77
“The time required to plan and implement a realistic simulation will usually be considerable, but in domains such as RNA sequencing, mature off-the-shelf simulators are available 75,151–153 .”
Self-citationConsiderations for RNA-seq Analysis of Circadian Rhythms
Li, J., Grant, G. R., Hogenesch, J. B., & Hughes, M. E. (2014) · Methods in enzymology on CD-ROM/Methods in enzymology · doi:10.1016/bs.mie.2014.10.020 · Cited by 54
Self-citationComparative evaluation of full-length isoform quantification from RNA-Seq
Sarantopoulou, D., Brooks, T. G., Nayak, S., Mrčela, A., Lahens, N. F., & Grant, G. R. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04198-1 · Cited by 59
“Hayer et al (Hayer et al., 2015) investigate such methods and conclude that none of the evaluated Sarantopoulou et al, Benchmarking of FLI quantification for RNA-Seq - 4 .”
BackgroundMethodSelf-citationRioux, G., Ridha, Z., Simard, M., Turgeon, F., Guérin, S. L., & Pouliot, R. (2020) · Genes · doi:10.3390/genes11101155 · Cited by 50
The hitchhikers’ guide to RNA sequencing and functional analysis
Chen, J.-W., Shrestha, L., Green, G. A., Leier, A., & Marquez‐Lago, T. T. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbac529 · Cited by 56
A survey on identification and quantification of alternative polyadenylation sites from RNA-seq data
Chen, M., Ji, G., Fu, H., Lin, Q., Ye, C., Ye, W., Su, Y., & Wu, X. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz068 · Cited by 36
Brown, J. W. S., Calixto, C. P. G., & Zhang, R. (2016) · New Phytologist · doi:10.1111/nph.14208 · Cited by 40
“The determination of transcripts from short read data is inaccurate, generates mis-assembled transcripts, does not assemble some bona fide transcripts, and different programs show variation in transcript assemblies (Steijger et al., 2013; Hayer et al., 2015; Pertea et al., 2015).”
BackgroundReference-free transcriptome exploration reveals novel RNAs for prostate cancer diagnosis
Pinskaya, M., Saci, Z., Gallopin, M., Gabriel, M., Nguyen, H. T. N., Firlej, V., Descrimes, M., Rapinat, A., Gentien, D., TAILLE, A. D. L., et al. (2019) · Life Science Alliance · doi:10.26508/lsa.201900449 · Cited by 36
“This complexity renders laborious in time and machine memory the data analysis by splice graph – based assemblers such as Cuf fl inks (Hayer et al, 2015; Kukurba & Montgomery,2015).”
BackgroundMethodA study of differential microRNA expression profile in migraine: the microMIG exploratory study
Gallardo, V. J., Gómez‐Galván, J. B., Asskour, L., Torres‐Ferrús, M., Alpuente, A., Caronna, E., & Pozo‐Rosich, P. (2023) · The Journal of Headache and Pain · doi:10.1186/s10194-023-01542-z · Cited by 31
Genome‐wide characterization of differential transcript usage in Arabidopsis thaliana
Vaneechoutte, D., Estrada, A. D., Lin, Y., Loraine, A. E., & Vandepoele, K. (2017) · The Plant Journal · doi:10.1111/tpj.13746 · Cited by 46
“However, the large number of isoforms per gene in AtRTD2 potentially complicates expression quantification as an increased number of isoforms negatively impacts quantification accuracy (Hayer et al., 2015; Kanitz et al., 2015; Zhang et al., 2017a).”
BackgroundTerminal exon characterization with TECtool reveals an abundance of cell-specific isoforms
Gruber, A. J., Gypas, F., Riba, A., Schmidt, R., & Zavolan, M. (2018) · Nature Methods · doi:10.1038/s41592-018-0114-z · Cited by 24
Ryūtō: network-flow based transcriptome reconstruction
Gatter, T., & Stadler, P. F. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2786-5 · Cited by 28
“For technical details on the datasets we refer to [35].”
InfluentialTheory and A Heuristic for the Minimum Path Flow Decomposition Problem
Shao, M., & Kingsford, C. (2017) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2017.2779509 · Cited by 27
Nanoribbon-Based Electronic Detection of a Glioma-Associated Circular miRNA
Ivanov, Y. D., Malsagova, K. A., Popov, V. P., Pleshakova, T. O., Kozlov, A. F., Galiullin, R. A., Shumov, I. D., Kapustina, S. I., Tikhonenko, F. V., Ziborov, V. S., et al. (2021) · Biosensors · doi:10.3390/bios11070237 · Cited by 23
“In this connection, it should be, however, emphasized that the implementation of RNA-seq requires considerable conceptual and computational improvements in order to increase the accuracy [24,25].”
BackgroundRefShannon: A genome-guided transcriptome assembler using sparse flow decomposition
Mao, S., Pachter, L. S., Tse, D. N. C., & Kannan, S. (2020) · PLoS ONE · doi:10.1371/journal.pone.0232946 · Cited by 20
“While algorithms and software packages are available for both the de novo (TransAByss [11], Trinity [12], OASES [13], SOAPdenovo-Trans [14] etc) and genome-guided assembly problems (Scripture [15], Cufflinks [16], StringTie [17], TransComb [18] and CLASS2 [19], Ryuto [20], Strawberry [21], Trinity (reference-guided mo…”
MethodAlternative splicing analysis benchmark with DICAST
Fenn, A. M., Tsoy, O. V., Faro, T., Rößler, F., Dietrich, A., Kersting, J., Louadi, Z., Lio, C. T., Völker, U., Baumbach, J., et al. (2022) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqad044 · Cited by 23
Challenges and prospects in utilizing technologies for gene fusion analysis in cancer diagnostics
Su, X., Zheng, Q., Xiu, X., Zhao, Q., Wang, Y., Han, D. W., & Song, P. (2024) · Med-X · doi:10.1007/s44258-024-00033-3 · Cited by 17
“As a result, several tools have been developed for detecting fusion transcripts from RNA-seq data, and many studies have compared these RNA-seq techniques [99–103].”
Omics approaches to study gene regulatory networks for development in echinoderms
Lowe, E. K., Cuomo, C., & Arnone, M. I. (2017) · Briefings in Functional Genomics · doi:10.1093/bfgp/elx012 · Cited by 15
Probing instructions for expression regulation in gene nucleotide compositions
Bessière, C., Taha, M., Petitprez, F., Vandel, J., Marin, J., Bréhélin, L., Lèbre, S., & Lecellier, C. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005921 · Cited by 17
“Yet another simple explanation could be that reconstructing and quantifying full-length mRNA transcripts is a difficult task, and no satisfying solution exists for now [53].”
BackgroundEvent Analysis: Using Transcript Events To Improve Estimates of Abundance in RNA-seq Data
Newman, J. R. B., CONCANNON, P. J., Tardáguila, M., Conesa, A., & McIntyre, L. M. (2018) · G3 Genes Genomes Genetics · doi:10.1534/g3.118.200373 · Cited by 14
“…not de novo assemble transcripts, as accuratetranscript reassembly and resolution from short-read data are known to result in a higher number of false transcripts (Steijger et al. 2013; Angelini et al. 2014; Bernard et al. 2015; Hayer et al. 2015; Liu et al. 2016a; Song et al. 2016) and shown here.”
BackgroundAssisted transcriptome reconstruction and splicing orthology
Blanquart, S., Varré, J., Guertin, P., Perrin, A., Bergeron, A., & Swenson, K. M. (2016) · BMC Genomics · doi:10.1186/s12864-016-3103-6 · Cited by 14
“Those findings were confirmed by several other studies [3–5], that include methods recently developed such as StringTie [6].”
ResultDIEGO: detection of differential alternative splicing using Aitchison’s geometry
Doose, G., Bernhart, S. H., Wagener, R., & Hoffman, S. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx690 · Cited by 10
Szkop, K. J., & Nobeli, I. (2017) · BioEssays · doi:10.1002/bies.201700090 · Cited by 9
Strategies and technologies for exploring long noncoding RNAs in heart failure
Zhang, Z., Wan, J., Liu, X., & Zhang, W. (2020) · Biomedicine & Pharmacotherapy · doi:10.1016/j.biopha.2020.110572 · Cited by 10
Caballero, J., Seoane, P., Jabato, F. M., Perkins, J. R., Manchado, M., & Claros, M. G. (2020) · Scientific Reports · doi:10.1038/s41598-020-77201-z · Cited by 12
Improved annotation with de novo transcriptome assembly in four social amoeba species
Singh, R., Lawal, H., Schilde, C., Glöckner, G., Barton, G. J., Schaap, P., & Cole, C. (2017) · BMC Genomics · doi:10.1186/s12864-017-3505-0 · Cited by 10
“Reference-free methods do not have any 56 dependencies, but need to deal with sequencing errors sufficiently well to avoid poor assemblies 57 [11-13].”
MethodGene-level differential analysis at transcript-level resolutionPreprint
Yi, L., Pimentel, H., Bray, N., & Pachter, L. S. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/190199 · Cited by 10
Evaluating the activity of nonsense-mediated RNA decay via Nanopore direct RNA sequencing
Li, X., Wan, L., Zhang, L., Zhuo, Z., Luo, X., Cui, J., Liu, Y., Su, F., Tang, M., & Xiao, F. (2022) · Biochemical and Biophysical Research Communications · doi:10.1016/j.bbrc.2022.06.055 · Cited by 8
Hartley, S. W., Mullikin, J. C., Klein, D. C., Park, M., Program, N. C. S., & Coon, S. L. (2016) · PLoS ONE · doi:10.1371/journal.pone.0163590 · Cited by 8
Validation of Splicing Events in Transcriptome Sequencing Data
Kaisers, W., Ptok, J., Schwender, H. R., & Schaal, H. (2017) · International Journal of Molecular Sciences · doi:10.3390/ijms18061110 · Cited by 9
“Currently, transcript reconstruction algorithms suffer from inaccuracies [1,2] and even the (much simpler) identification of splice events is associated with a high false discovery rate (FDR) [3].”
BackgroundSchou, M. F., Bechsgaard, J., Muñoz, J., & Kristensen, T. N. (2018) · Evolution · doi:10.1111/evo.13497 · Cited by 10
“Novel transcript assemblies should be treated with care (Hayer et al. 2015), but given the extreme heat stress during development (31°C) and the potential for inbreeding affecting splicing, we found it of biological relevance to allow the mapping of novel transcripts of high confidence within…”
BackgroundHou, Y., Li, Q., Zhou, H., Kafle, S., Li, W., Tan, L., Liang, J., Meng, L., & Xin, H. (2024) · Plant Physiology and Biochemistry · doi:10.1016/j.plaphy.2024.108863 · Cited by 7
The contribution of uncharted RNA sequences to tumor identity in lung adenocarcinoma
Wang, Y., Xue, H., Aglave, M., Lainé, A., Gallopin, M., & Gautheret, D. (2022) · NAR Cancer · doi:10.1093/narcan/zcac001 · Cited by 6
McSplicer: a probabilistic model for estimating splice site usage from RNA-seq data
Alqassem, I., Sonthalia, Y., Klitzke-Feser, E., Shim, H., & Canzar, S. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab050 · Cited by 7
“It assigns weights to the edges of a splicing graph (Heber et al., 2002) using parameters that describe the splicing process, rather than focusing on individual outcomes of the splicing processes such as local splicing events or full-length transcripts. The parameter estimates can then be used to estimate transcript a…”
MethodCAMPAREE: a robust and configurable RNA expression simulator
Lahens, N. F., Brooks, T. G., Sarantopoulou, D., Nayak, S., Lawrence, C., Mrčela, A., Srinivasan, A., Schug, J., Hogenesch, J. B., Barash, Y., et al. (2021) · BMC Genomics · doi:10.1186/s12864-021-07934-2 · Cited by 6
“BEERS simulates variants, errors, variable intron signal, and non-uniform sequence coverage, and despite being originally designed to benchmark aligners has since been adapted for use with transcript reconstruction and alternative splicing analyses [10, 11].”
MethodSelf-citationKara, M. F., Guo, W., Zhang, R., & Denby, K. (2024) · The Plant Journal · doi:10.1111/tpj.16978 · Cited by 4
Meng, R., Li, Z., Kang, X., Zhang, Y., Wang, Y., Ma, Y., Wu, Y., Dong, S., Li, X., Gao, L., et al. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25115840 · Cited by 3
“…lead to differences in the expression In addition, some studies have shown that the presence of multiple transcripts can a ff ect RNA-seq results, as the presence of di ff erent transcripts can lead to di ff erences in the expression levels of the same gene [35–37]. levels of the same gene [35–37].”
BackgroundClusTrast: a short read de novo transcript isoform assembler guided by clustered contigs
Westrin, K. J., Kretzschmar, W. W., & Emanuelsson, O. (2024) · BMC Bioinformatics · doi:10.1186/s12859-024-05663-3 · Cited by 4
Transcriptomic signatures across a critical sedimentation threshold in a major reef-building coral
Lock, C., Gabriel, M. M., & Bentlage, B. (2024) · Frontiers in Physiology · doi:10.3389/fphys.2024.1303681 · Cited by 6
Wang, D., Gazzara, M. R., Jewell, S., Wales-McGrath, B. D., Yang, K., Brown, C. D., Choi, P. S., & Barash, Y. (2025) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2025.10.012 · Cited by 3
“Although several studies highlighted the limited accuracy of whole-transcript quantification from short-read RNA-seq data, numerous studies have used this approach for transcript QTLs (trQTLs), 18,33 so assessing the value of this approach is warranted.”
Majoros, W. H., Campbell, M. S., Holt, C. H., DeNardo, E. K., Ware, D., Allen, A. S., Yandell, M. D., & Reddy, T. E. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw799 · Cited by 2
“…reviewed in Majoros, 2007; Stanke et al., 2006) and applied to the problem of annotating reference genomes, both human and non-human (Adams et al., 2000; Haas et al., 2008; Holt and Yandell, 2011; Lander et al., 2001; Parra et al., 2007; Venter et al., 2001; reviewed in Yandell and Ence, 2012).”
BackgroundHernandez-Lopez, A. A., Voges, J., Alberti, C., Mattavelli, M., & Östermann, J. (2018) · DCC · doi:10.1109/dcc.2018.00025 · Cited by 5
“According to [21] the best performing tools for this task are Cufflinks [22] and StringTie [23].”
MethodSRSF shape analysis for sequencing data reveal new differentiating patterns
Wesołowski, S., Vera, D. L., & Wu, W. (2017) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2017.07.004 · Cited by 2
Performance evaluation of lossy quality compression algorithms for RNA-seq data
Yu, R., Yang, W., & Wang, S. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-03658-4 · Cited by 2
“In this paper, we provide a systematic evaluation on the impacts of lossy quality compression on common RNA-seq data analysis pipelines including gene expression quantification [13], transcript alternative splicing [14, 15] and short variants detection [16, 17].”
MethodExperimental Design for Time-Series RNA-Seq Analysis of Gene Expression and Alternative Splicing
Τzioutziou, N. A., James, A. B., Guo, W., Calixto, C. P. G., Zhang, R., Nimmo, H. G., & Brown, J. W. S. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1912-4_14 · Cited by 2
Scavenger: A pipeline for recovery of unaligned reads utilising similarity with aligned reads
Yang, A., Tang, J. Y. S., Troup, M., & Ho, J. W. K. (2019) · F1000Research · doi:10.12688/f1000research.19426.1 · Cited by 3
“However, these alignment-free tools are only applicable to specific types of analyses and have limitations compared to traditional alignment methods 7 .”
MethodGenome-wide characterization of differential transcript usage in Arabidopsis thalianaPreprint
Vaneechoutte, D., Estrada, A. R., Lin, Y.-C., Loraine, A. E., & Vandepoele, K. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/136770 · Cited by 2
“However, the large number of isoforms per gene in AtRTD2 potentially complicates expression quantification since an increased number of isoforms negatively impacts quantification accuracy (Hayer et al., 2015; Kanitz et al., 2015; Zhang et al., 2017).”
BackgroundUse of short-read RNA-Seq data to identify transcripts that can translate novel ORFsPreprint
Erady, C., Puntambekar, S. S., & Prabakaran, S. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.03.21.001883 · Cited by 2
“Moreover, RSEM and StringTie are well-cited tools and have shown to be better than other assemblers for transcript assembly, especially lncRNAs, which is an important category in our investigation 38–40 .”
BackgroundMethodMASQ: A multiplex qPCR platform for quantitative alternative splicing analysis in clinical specimens
Jeong, H., Cho, N., Park, J.-I., Yoo, H. J., & Kim, K. K. (2026) · Molecular Therapy — Nucleic Acids · doi:10.1016/j.omtn.2026.102923 · Cited by 1
BOAssembler: A Bayesian Optimization Framework to Improve RNA-Seq Assembly Performance
Mao, S., Jiang, Y., Mathew, E. B., & Kannan, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-42266-0_15 · Cited by 1
Efficient Heuristic for Decomposing a Flow with Minimum Number of PathsPreprint
Shao, M., & Kingsford, C. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/087759 · Cited by 2
“However, according to the benchmarking studies [18, 19], the performance of these methods are far from satisfactory, especially when the expressed genes contain multiple splice isoforms.”
BackgroundClusTrast: a short read de novo transcript isoform assembler guided by clustered contigsPreprint
Westrin, K. J., Kretzschmar, W. W., & Emanuelsson, O. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.01.02.473666 · Cited by 1
High-throughput RNA sequencing: a step forward in transcriptome analysis
Okonechnikov, K. (2016) · Universitätsbibliothek der FU Berlin Hochschulschriftenstelle u. Dokumentenserver · doi:10.17169/refubium-13309 · Cited by 1
Analysis of Long Noncoding RNAs in RNA-Seq Data
Niazi, F., & Valadkhan, S. (2016) · Springer eBooks · doi:10.1007/978-3-319-31350-4_7
Guarischi‐Sousa, R. (2017) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.95.2017.tde-28092017-112917
Transcriptome-based identification of small RNA in plants: The need for robust prediction algorithms
Ganguli, S., Singh, P. K., & Pal, A. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-817112-7.00004-3 · Cited by 1
McSplicer: a probabilistic model for estimating splice site usage from RNA-seq dataPreprint
Alqassem, I., Sonthalia, Y., Klitzke-Feser, E., Shim, H., & Canzar, S. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.08.10.243097
Schubert, D. S. P. (2019) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.733234891.793557089
Towards reproducible, transparent, and systematic benchmarking of omics computational toolsPreprint
Mangul, S., Martin, L. S., Hill, B., Lam, A. K.-M., DISTLER, M., Zelikovsky, A., Eskin, E., & Flint, J. (2018) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/p8yd9
Investigating Alzheimer’s Disease-Associated Genes Using Differential Splicing Frequency Analysis
Yao, Y., Cheng, Z., Chen, S., Shi, J., Zhang, Y., Liu, C., Wei, D., Zhang, T., Duan, G., & Gao, S. (2026) · Cells · doi:10.3390/cells15121086 · Cited by 1
“A previous study [2] concluded that none of the evaluated methods is accurate enough for de novo transcriptome assembly, where isoform structures must be inferred directly from the RNA-Seq data.”
Westrin, K. J., Kretzschmar, W. W., & Emanuelsson, O. (2024) · Figshare · doi:10.6084/m9.figshare.26675579
Probing sequence-level instructions for gene expression
Taha, M. (2018) · theses.fr (ABES) · doi:10.70675/4af494d2z4e2ez49ccz9821ze38a2c283ad4
Nguyen, T. N. H. (2020) · theses.fr (ABES) · doi:10.70675/10659a6dz4064z4c7czae0fz0f0373fbaab9 · Cited by 2
Etude des éléments régulateurs de l'expression des gènes chez l'humain
Bessière, C. (2018) · theses.fr (ABES) · doi:10.70675/0b26676ez90bfz4f47zba59z90853f19d510
k-mer based analysis for cancer transcriptomics at nucleotide resolution
Xue, H. (2021) · theses.fr (ABES) · doi:10.70675/0ea2dc79z93a0z4759z98d7z86877625034f
Guo, W., Milne, L., López-Gomollón, S., Kaur, A., Hein, I., Baulcombe, D. C., & Zhang, R. (2026) · Scientific Data · doi:10.1038/s41597-026-07792-1
“…up to 50% of the transcripts produced directly by these tools, particularly transcript assemblies based on short reads, could be mis-assembled 12,13 , Subsequently, these are manifested as transcripts containing false splice junctions, incomplete transcript fragments, or transcripts with…”
Metadata
- Authors
- Hayer, K. E., Pizarro, A., Lahens, N. F., Hogenesch, J. B., & Grant, G. R.
- Affiliations
- University of Pennsylvania (US)
- Amazon (United States) (US)
- 3 Department of Pharmacology and
- Corresponding author
- Hayer, Katharina E.
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2015-09-03
- Volume
- 31
- Issue
- 24
- Pages
- 3938-3945
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC4673975
- OpenAlex
- W2106352274
- Cited by
- 93
- References
- 35
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 8 · 2023: 2 · 2022: 5 · 2021: 8 · 2020: 14 · 2019: 8 · 2018: 12 · 2017: 16 · 2016: 12 · 2015: 2
- FWCI
- 6.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Bioinformatics (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Funders
- Pennsylvania Department of Health
- University of Pennsylvania
- Institute for Translational Medicine and Therapeutics
- National Institutes of Health (UL1TR000003)
- Defense Advanced Research Projects Agency (DARPA-D12AP00025, D12AP00025)
- Advanced Research Projects Agency
- National Heart, Lung, and Blood Institute (U54HL117798)
- National Institute of Neurological Disorders and Stroke (R01NS054794)
- National Institute of Environmental Health Sciences (P30ES013508)
- National Center for Advancing Translational Sciences
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA Research and Splicing
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Keywords
- alternative splicing, RNA-seq, benchmarking, sequencing errors, polymorphism, alignment errors, annotation errors, algorithm accuracy
- MeSH terms
- Algorithms, Animals, Humans, RNA, Messenger, Alternative Splicing, Sequence Analysis, RNA, Benchmarking, Gene Expression Profiling, Mice, RNA Isoforms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.