Biologically relevant transfer learning improves transcription factor binding prediction
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Background: Deep learning has proven to be a powerful technique for transcription factor (TF) binding prediction but requires large training datasets. Transfer learning can reduce the amount of data required for deep learning, while improving overall model performance, compared to training a separate model for each new task. Results: We assess a transfer learning strategy for TF binding prediction consisting of a pre-training step, wherein we train a multi-task model with multiple TFs, and a fine-tuning step, wherein we initialize single-task models for individual TFs with the weights learned by the multi-task model, after which the single-task models are trained at a lower learning rate. We corroborate that transfer learning improves model performance, especially if in the pre-training step the multi-task model is trained with biologically relevant TFs. We show the effectiveness of transfer learning for TFs with ~ 500 ChIP-seq peak regions. Using model interpretation techniques, we demonstrate that the features learned in the pre-training step are refined in the fine-tuning step to resemble the binding motif of the target TF (i.e., the recipient of transfer learning in the fine-tuning step). Moreover, pre-training with biologically relevant TFs allows single-task models in the fine-tuning step to learn useful features other than the motif of the target TF. Conclusions: Our results confirm that transfer learning is a powerful technique for TF binding prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Chicco, D., & Jurman, G. (2020) · BMC Genomics · doi:10.1186/s12864-019-6413-7 · Cited by 6,143 · cited 1×
“Given the advantages of the Matthews correlation coefficient (MCC) over other more popular metrics for evaluating binary classification tasks [43], we chose MCC as our performance metric.”
MethodBiopython: freely available python tools for computational molecular biology and bioinformatics
Cock, P. J. A., Antao, T., Chang, J. T., Chapman, B. A., Cox, C. J., Dalke, A., Friedberg, I., Hamelryck, T., Kauff, F., Wilczynski, B., et al. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp163 · Cited by 6,080
Learning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., & Kundaje, A. (2017) · arXiv · doi:10.48550/arxiv.1704.02685 · Cited by 2,761
Quantifying similarity between motifs
Gupta, S., Stamatoyannopoulos, J. A., Bailey, T. L., & Noble, W. S. (2007) · Genome Biology · doi:10.1186/gb-2007-8-2-r24 · Cited by 2,058
JASPAR 2020: update of the open-access database of transcription factor binding profiles
Fornes, O., Castro-Mondragon, J. A., Khan, A., van der Lee, R., Zhang, X., Richmond, P. A., Modi, B. P., Correard, S., Gheorghe, M., Baranašić, D., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz1001 · Cited by 1,763
Self-citationKelley, D. R., Snoek, J., & Rinn, J. L. (2016) · Genome Research · doi:10.1101/gr.200535.115 · Cited by 1,048
Visualizing and Understanding Convolutional Networks
Zeiler, M. D., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1311.2901 · Cited by 522
Highly expressed loci are vulnerable to misleading ChIP localization of multiple unrelated proteins
Teytelman, L., Thurtle, D. M., Rine, J., & van Oudenaarden, A. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1316064110 · Cited by 389
Shrikumar, A., Tian, K., Avsec, Ž., Shcherbina, A., Banerjee, A., Sharmin, M., Nair, S., & Kundaje, A. (2018) · arXiv · doi:10.48550/arxiv.1811.00416 · Cited by 229
UCSC Genome Browser enters 20th year
Lee, C. M., Barber, G. P., Casper, J., Clawson, H., Diekhans, M., Gonzalez, J. N., Hinrichs, A. S., Lee, B. T., Nassar, L. R., Powell, C. C., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz1012 · Cited by 191
Sharing and Specificity of Co-expression Networks across 35 Human Tissues
Pierson, E., the GTEx Consortium, Koller, D., Battle, A., & Mostafavi, S. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004220 · Cited by 177
Self-citationNon-targeted transcription factors motifs are a systemic component of ChIP-seq datasets
Worsley Hunt, R., & Wasserman, W. W. (2014) · Genome Biology · doi:10.1186/s13059-014-0412-4 · Cited by 73
Self-citation
Cited by
Multimodal data fusion for cancer biomarker discovery with deep learning
Steyaert, S., Pizurica, M., Nagaraj, D., Khandelwal, P., Hernandez-Boussard, T., Gentles, A. J., & Gevaert, O. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00633-5 · Cited by 402
“…include basic data transformations as well as generation of synthetic data, but other strategies such as semi-supervised learning 136–139 , active learning 140,141 , transfer learning 139,142–144 and automated annotation 145,146 have shown to be promising avenues to overcome labelled-data scarcity.”
BackgroundTargeted design of synthetic enhancers for selected tissues in the Drosophila embryo
de Almeida, B. P., Schaub, C., Pagani, M., Secchia, S., Furlong, E. E. M., & Stark, A. (2024) · Nature · doi:10.1038/s41586-023-06905-9 · Cited by 87
Naqvi, S., Kim, S., Tabatabaee, S., Pampari, A., Kundaje, A., Pritchard, J. K., & Wysocka, J. (2025) · Cell Genomics · doi:10.1016/j.xgen.2025.100780 · Cited by 18
Salvatore, M., Horlacher, M., Marsico, A., Winther, O., & Andersson, R. (2023) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqad026 · Cited by 13
Metadata
- Authors
- Novakovsky, G., Saraswat, M., Fornes, O., Mostafavi, S., & Wasserman, W. W.
- Venue
- Genome Biology
- Published
- 2021-09-27
- Volume
- 22
- Issue
- 1
- Pages
- 280-280
- ISSN
- 1465-6906 (ISSN-L)
- 1474-760X
- OpenAlex
- W3204042176
- Cited by
- 44
- References
- 13
- FWCI
- 2.35
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.