The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Benchmark analysis of algorithms for determining and quantifying full-length mRNA splice forms from RNA-seq data
Open accessPublished version of a preprint
Abstract
Motivation: Because of the advantages of RNA sequencing (RNA-Seq) over microarrays, it is gaining widespread popularity for highly parallel gene expression analysis. For example, RNA-Seq is expected to be able to provide accurate identification and quantification of full-length splice forms. A number of informatics packages have been developed for this purpose, but short reads make it a difficult problem in principle. Sequencing error and polymorphisms add further complications. It has become necessary to perform studies to determine which algorithms perform best and which if any algorithms perform adequately. However, there is a dearth of independent and unbiased benchmarking studies. Here we take an approach using both simulated and experimental benchmark data to evaluate their accuracy. Results: We conclude that most methods are inaccurate even using idealized data, and that no method is highly accurate once multiple splice forms, polymorphisms, intron signal, sequencing errors, alignment errors, annotation errors and other complicating factors are present. These results point to the pressing need for further algorithm development. Availability and implementation: Simulated datasets and other supporting information can be found at http://bioinf.itmat.upenn.edu/BEERS/bp2 Supplementary information: Supplementary data are available at Bioinformatics online. Contact: hayer@upenn.edu
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Even so, it is common to observe extreme variability in the coverage of RNA-seq fragments along transcripts that is purely technical and sample-specific17 and not explained by current bias models, which confounds current methods designed to identify and quantify transcripts18”
From Modeling of RNA-seq fragment sequence bias reduces systematic errors in transcript abundance estimation (2016) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STAR: ultrafast universal RNA-seq aligner
Dobin, A., Davis, C. A., Schlesinger, F., Drenkow, J., Zaleski, C., Jha, S., Batut, P., Chaisson, M., & Gingeras, T. R. (2013) · Bioinformatics · doi:10.1093/bioinformatics/bts635 · Cited by 51,709 · cited 1×
“For alignment-guided methods, we aligned the reads with TopHat2 [20] (hereafter referred to as TopHat) and STAR [21].”
RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome
Li, B., & Dewey, C. N. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-323 · Cited by 22,179 · cited 2×
“, 2015), RSEM (Li and Dewey, 2011) and IsoEM (Nicolae et al.”
BackgroundHTSeq—a Python framework to work with high-throughput sequencing data
Anders, S., Pyl, P. T., & Huber, W. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btu638 · Cited by 21,464 · cited 1×
“These results point to the need for new algorithm development to solve this difficult, but important, problem.”
BackgroundFull-length transcriptome assembly from RNA-Seq data without a reference genome
Grabherr, M. G., Haas, B. J., Yassour, M., Levin, J. Z., Thompson, D. A., Amit, I., Adiconis, X., Fan, L., Raychowdhury, R., Zeng, Q., et al. (2011) · Nature Biotechnology · doi:10.1038/nbt.1883 · Cited by 21,039
Trapnell, C., Williams, B. A., Pertea, G., Mortazavi, A., Kwan, G., van Baren, M. J., Salzberg, S. L., Wold, B. J., & Pachter, L. (2010) · Nature Biotechnology · doi:10.1038/nbt.1621 · Cited by 15,556
StringTie enables improved reconstruction of a transcriptome from RNA-seq reads
Pertea, M., Pertea, G. M., Antonescu, C. M., Chang, T.-C., Mendell, J. T., & Salzberg, S. L. (2015) · Nature Biotechnology · doi:10.1038/nbt.3122 · Cited by 14,298 · cited 1×
“They assessed performance by correlation; however that does not account for the accuracy of absolute intensity levels, just how the values co-vary.”
BackgroundKim, D., Pertea, G., Trapnell, C., Pimentel, H., Kelley, R., & Salzberg, S. L. (2013) · Genome Biology · doi:10.1186/gb-2013-14-4-r36 · Cited by 12,668 · cited 1×
“For alignment-guided methods, we aligned the reads with TopHat2 [20] (hereafter referred to as TopHat) and STAR [21].”
MethodThe Human Genome Browser at UCSC
Kent, W. J., Sugnet, C. W., Furey, T. S., Roskin, K. M., Pringle, T. H., Zahler, A. M., & Haussler, A. D. (2002) · Genome Research · doi:10.1101/gr.229102 · Cited by 10,400 · cited 1×
“BEERS starts with a genome and a set of gene annotations; here we used mouse genome build mm9 [9] and RefSeq genes [10], as aligned to the reference genome by the UCSC genome browser [11].”
MethodInitial sequencing and comparative analysis of the mouse genome
Mouse Genome Sequencing Consortium (2002) · Nature · doi:10.1038/nature01262 · Cited by 7,187
Differential analysis of gene regulation at transcript resolution with RNA-seq
Trapnell, C., Hendrickson, D. G., Sauvageau, M., Goff, L., Rinn, J. L., & Pachter, L. (2013) · Nature Biotechnology · doi:10.1038/nbt.2450 · Cited by 3,492 · cited 1×
“These results point to the need for new algorithm development to solve this difficult, but important, problem.”
Background
Cited by
Detecting circular RNAs: bioinformatic and experimental challenges
Szabo, L., & Salzman, J. (2016) · Nature Reviews Genetics · doi:10.1038/nrg.2016.114 · Cited by 683
“However, as has already been noted with respect to linear splice detection, increased coverage improves sensitivity but actually results in a higher false-positive rate 36,53,56 , and longer reads do not sufficiently address this issue 56 .”
BackgroundResultPetryszak, R., Keays, M., Tang, Y. A., Fonseca, N. A., Barrera, E., Burdett, T., Füllgrabe, A., Fuentes, A. M.-P., Jupp, S., Koskinen, S., et al. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1045 · Cited by 530
Modular, efficient and constant-memory single-cell RNA-seq preprocessing
Melsted, P., Booeshaghi, A. S., Liu, L., Gao, F., Lu, L., Min, K. H., da Veiga Beltrame, E., Hjörleifsson, K. E., Gehring, J., & Pachter, L. (2021) · Nature Biotechnology · doi:10.1038/s41587-021-00870-2 · Cited by 510
Sahraeian, S. M. E., Mohiyuddin, M., Sebra, R., Tilgner, H., Afshar, P. T., Au, K. F., Bani Asadi, N., Gerstein, M. B., Wong, W. H., Snyder, M. P., et al. (2017) · Nature Communications · doi:10.1038/s41467-017-00050-4 · Cited by 376
“On MCF7-100 and MCF7-300 samples, Cuf fl inks-TopHat prediction sets were not enriched in any MCF7 or breast cancer-related gene expression study, while StringTie-HISAT2 ’ s and Salmon-SMEM ’ s top over-expressed genes were highly enriched in many MCF7 and breast cancer cell line-related gene sets (Supplementary Data 1…”
BackgroundMethodInfluentialUnderstanding sequencing data as compositions: an outlook and review
Quinn, T. P., Erb, I., Richardson, M. F., & Crowley, T. M. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty175 · Cited by 350
“, [17]; [54]; [63]; [26]; [35]; [59]; [61]; [65]; [49]).”
Metadata
- Authors
- Hayer, K. E., Pizarro, A., Lahens, N. F., Hogenesch, J. B., & Grant, G. R.
- Affiliations
- University of Pennsylvania (US)
- Amazon (United States) (US)
- 3 Department of Pharmacology and
- Corresponding author
- Hayer, Katharina E.
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2015-09-03
- Volume
- 31
- Issue
- 24
- Pages
- 3938-3945
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC4673975
- OpenAlex
- W2106352274
- Cited by
- 93
- References
- 35
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 8 · 2023: 2 · 2022: 5 · 2021: 8 · 2020: 14 · 2019: 8 · 2018: 12 · 2017: 16 · 2016: 12 · 2015: 2
- FWCI
- 6.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Bioinformatics (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Funders
- Pennsylvania Department of Health
- University of Pennsylvania
- Institute for Translational Medicine and Therapeutics
- National Institutes of Health (UL1TR000003)
- Defense Advanced Research Projects Agency (DARPA-D12AP00025, D12AP00025)
- Advanced Research Projects Agency
- National Heart, Lung, and Blood Institute (U54HL117798)
- National Institute of Neurological Disorders and Stroke (R01NS054794)
- National Institute of Environmental Health Sciences (P30ES013508)
- National Center for Advancing Translational Sciences
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA Research and Splicing
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Keywords
- alternative splicing, RNA-seq, benchmarking, sequencing errors, polymorphism, alignment errors, annotation errors, algorithm accuracy
- MeSH terms
- Algorithms, Animals, Humans, RNA, Messenger, Alternative Splicing, Sequence Analysis, RNA, Benchmarking, Gene Expression Profiling, Mice, RNA Isoforms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Benchmark analysis of algorithms for determining and quantifying full-length mRNA splice forms from RNA-seq data. https://app.cochat.ai/papers/10.1093/bioinformatics/btv488
- The paper
- Hayer, K. E., Pizarro, A., Lahens, N. F., Hogenesch, J. B., & Grant, G. R. (2015). Benchmark analysis of algorithms for determining and quantifying full-length mRNA splice forms from RNA-seq data. Bioinformatics. https://doi.org/10.1093/bioinformatics/btv488
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.