Comprehensive evaluation of deep learning architectures for prediction of DNA/RNA sequence binding specificities
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
Motivation: Deep learning architectures have recently demonstrated their power in predicting DNA- and RNA-binding specificity. Existing methods fall into three classes: Some are based on convolutional neural networks (CNNs), others use recurrent neural networks (RNNs) and others rely on hybrid architectures combining CNNs and RNNs. However, based on existing studies the relative merit of the various architectures remains unclear. Results: In this study we present a systematic exploration of deep learning architectures for predicting DNA- and RNA-binding specificity. For this purpose, we present deepRAM, an end-to-end deep learning tool that provides an implementation of a wide selection of architectures; its fully automatic model selection procedure allows us to perform a fair and unbiased comparison of deep learning architectures. We find that deeper more complex architectures provide a clear advantage with sufficient training data, and that hybrid CNN/RNN architectures outperform other methods in terms of accuracy. Our work provides guidelines that can assist the practitioner in choosing an appropriate network architecture, and provides insight on the difference between the models learned by convolutional and recurrent networks. In particular, we find that although recurrent networks improve model accuracy, this comes at the expense of a loss in the interpretability of the features learned by the model. Availability and implementation: The source code for deepRAM is available at https://github.com/MedChaabane/deepRAM. Supplementary information: Supplementary data are available at Bioinformatics online.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · cited 1×
“In recent years, deep neural networks have become the technique of choice for challenging tasks in computer vision (Krizhevsky et al. 2012, LeCun et al. 2015), speech recognition (Hinton et al. 2012), machine translation (Sutskever et al. 2014), and computational biology (Angermueller et al. 2016).”
MethodLearning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738 · cited 1×
“Dilated convolution uses filters with gaps to allow each filter to capture information across larger and larger stretches of the input sequence (Yu and Koltun 2015).”
BackgroundPredicting the sequence specificities of DNA- and RNA-binding proteins by deep learning
Alipanahi, B., Delong, A., Weirauch, M. T., & Frey, B. J. (2015) · Nature Biotechnology · doi:10.1038/nbt.3300 · Cited by 2,873
Predicting effects of noncoding variants with deep learning–based sequence model
Zhou, J., & Troyanskaya, O. G. (2015) · Nature Methods · doi:10.1038/nmeth.3547 · Cited by 2,338
Quantifying similarity between motifs
Gupta, S., Stamatoyannopoulos, J. A., Bailey, T. L., & Noble, W. S. (2007) · Genome Biology · doi:10.1186/gb-2007-8-2-r24 · Cited by 2,058 · cited 2×
“Once the model is trained, the learned motifs of the first convolutional module are automatically extracted and visualized using Weblogo, and then matched with known motifs using TOMTOM.”
MethodA census of human RNA-binding proteins
Gerstberger, S., Hafner, M., & Tuschl, T. (2014) · Nature Reviews Genetics · doi:10.1038/nrg3813 · Cited by 2,032
Advances in natural language processing
Hirschberg, J., & Manning, C. D. (2015) · Science · doi:10.1126/science.aaa8685 · Cited by 1,712 · cited 2×
“This internal memory allows RNNs to capture interactions between distant elements along the sequence, and are therefore commonly used in natural language processing (Hirschberg and Manning 2015).”
BackgroundA compendium of RNA-binding motifs for decoding gene regulation
Ray, D., Kazan, H., Cook, K. B., Weirauch, M. T., Najafabadi, H. S., Li, X., Gueroussov, S., Albu, M., Zheng, H., Yang, A., et al. (2013) · Nature · doi:10.1038/nature12311 · Cited by 1,480
Deep learning for computational biology
Angermueller, C., Pärnamaa, T., Parts, L., & Stegle, O. (2016) · Molecular Systems Biology · doi:10.15252/msb.20156651 · Cited by 1,473 · cited 1×
“In recent years, deep neural networks have become the technique of choice for challenging tasks in computer vision (Krizhevsky et al. 2012, LeCun et al. 2015), speech recognition (Hinton et al. 2012), machine translation (Sutskever et al. 2014), and computational biology (Angermueller et al. 2016).”
MethodGrossberg, S. (2013) · Scholarpedia · doi:10.4249/scholarpedia.1888 · Cited by 602
Sequential regulatory activity prediction across chromosomes with convolutional neural networks
Kelley, D. R., Reshef, Y. A., Bileschi, M., Belanger, D., McLean, C. Y., & Snoek, J. (2018) · Genome Research · doi:10.1101/gr.227819.117 · Cited by 537
Protein-DNA binding: complexities and multi-protein codes
Siggers, T., & Gordân, R. (2014) · Nucleic Acids Research · doi:10.1093/nar/gkt1112 · Cited by 229 · cited 1×
“In order to represent binding sites and detect new ones, Position Weight Matrices (PWMs) are the most common method to characterize the sequence specificity of a protein thanks to their simplicity and ease of interpretation (Stormo 2000).”
Method
Cited by
Zrimec, J., Börlin, C. S., Buric, F., Muhammad, A. S., Chen, R., Siewers, V., Verendel, V., Nielsen, J., Töpel, M., & Zelezniak, A. (2020) · Nature Communications · doi:10.1038/s41467-020-19921-4 · Cited by 174
Gene expression is encoded in all parts of a co-evolving interacting gene regulatory structure
Zrimec, J., Buric, F., Muhammad, A. S., Chen, R., Verendel, V., Töpel, M., & Zelezniak, A. (2019) · doi:10.1101/792531 · Cited by 173
Tackling prediction uncertainty in machine learning for healthcare
Chua, M., Kim, D., Choi, J., Lee, N. G., Deshpande, V., Schwab, J., Lev, M. H., Gonzalez, R. G., Gee, M. S., & Do, S. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00988-x · Cited by 114
Yuan, H., Yu, K., Xie, F., Liu, M., & Sun, S. (2024) · Medicine Advances · doi:10.1002/med4.75 · Cited by 61
Benchmarking deep learning methods for predicting CRISPR/Cas9 sgRNA on- and off-target activities
Zhang, G., Luo, Y., Dai, X., & Dai, Z. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad333 · Cited by 47
Metadata
- Authors
- Trabelsi, A., Chaabane, M., & Ben-Hur, A.
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2019-07
- Volume
- 35
- Issue
- 14
- Pages
- i269-i277
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- OpenAlex
- W2918598256
- Cited by
- 153
- References
- 18
- FWCI
- 6.09
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.