Assessing computational tools for the discovery of transcription factor binding sites
Open accessAbstract
The prediction of regulatory elements is a problem where computational methods offer great hope. Over the past few years, numerous tools have become available for this task. The purpose of the current assessment is twofold: to provide some guidance to users regarding the accuracy of currently available tools in various settings, and to provide a benchmark of data sets for assessing future tools.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Computational approaches to predict physical interactions can partly circumvent bias but do not address tissue specificity and may have relatively low reproducibility across methods.
“Computational approaches to predict physical interactions can partly circumvent bias (TABLE 1), but they do not address tissue specificity, and there may be relatively low reproducibility across different methods 58,59.”
From Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders (2015) · cites this paper as [59]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Prediction of complete gene structures in human genomic DNA
Burge, C., & Karlin, S. (1997) · Journal of Molecular Biology · doi:10.1006/jmbi.1997.0951 · Cited by 4,211 · see citing works
Hertz, G. Z., & Stormo, G. D. (1999) · Bioinformatics · doi:10.1093/bioinformatics/15.7.563 · Cited by 1,246 · see citing works
TRANSFAC: a database on transcription factors and their DNA binding sites
Wingender, E. (1996) · Nucleic Acids Research · doi:10.1093/nar/24.1.238 · Cited by 1,064 · see citing works
Hughes, J. D., Estep, P. W., Tavazoie, S., & Church, G. M. (2000) · Journal of Molecular Biology · doi:10.1006/jmbi.2000.3519 · Cited by 1,008 · see citing works
Self-citationvan Helden, J., André, B., & Collado‐Vides, J. (1998) · Journal of Molecular Biology · doi:10.1006/jmbi.1998.1947 · Cited by 748 · see citing works
Evaluation of Gene Structure Prediction Programs
Burset, M., & Guigó, R. (1996) · Genomics · doi:10.1006/geno.1996.0298 · Cited by 741 · see citing works
The value of prior knowledge in discovering motifs with MEME.
Bailey, T. L., & Elkan, C. (1995) · PubMed · Cited by 665 · see citing works
Self-citationCombinatorial approaches to finding subtle signals in DNA sequences.
Pevzner, P. A., & Sze, S. (2000) · PubMed · Cited by 547 · see citing works
Pavesi, G., Mereghetti, P., Mauri, G., & Pesole, G. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh465 · Cited by 491 · see citing works
Self-citationThijs, G., Lescot, M., Kathleen Marchal, D., Rombauts, S., Moor, B. D., Rouzé, P., & Moreau, Y. (2001) · Bioinformatics · doi:10.1093/bioinformatics/17.12.1113 · Cited by 393 · see citing works
Self-citationFinding composite regulatory patterns in DNA sequences
Eskin, E., & Pevzner, P. A. (2002) · Bioinformatics · doi:10.1093/bioinformatics/18.suppl_1.s354 · Cited by 332 · see citing works
Self-citationDiscovering regulatory elements in non-coding sequences by analysis of spaced dyads
van Helden, J. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.8.1808 · Cited by 316 · see citing works
Sinha, S. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg618 · Cited by 257 · see citing works
Self-citationANN-SPEC: A METHOD FOR DISCOVERING TRANSCRIPTION FACTOR BINDING SITES WITH IMPROVED SPECIFICITY
Workman, C. T., & Stormo, G. D. (1999) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789814447331_0044 · Cited by 238 · see citing works
Self-citationCritical assessment of methods of protein structure prediction (CASP)-round V
Moult, J., Fidelis, K. A., Zemla, A. T., & Hubbard, T. (2003) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.10556 · Cited by 228 · see citing works
Genome Annotation Assessment in Drosophila melanogasterPreprint
Reese, M. G., Hartzell, G., Harris, N. L., Ohler, U., Abril, J. F., & Lewis, S. (2000) · Genome Research · doi:10.1101/gr.10.4.483 · Cited by 207 · see citing works
Finding functional sequence elements by multiple local alignment
Frith, M. C. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh169 · Cited by 198 · see citing works
Self-citationPhyME: A probabilistic algorithm for finding motifs in sets of orthologous sequences
Sinha, S., Blanchette, M., & Tompa, M. (2004) · BMC Bioinformatics · doi:10.1186/1471-2105-5-170 · Cited by 197 · see citing works
Self-citationEnvironmentally Induced Foregut Remodeling by PHA-4/FoxA and DAF-12/NHR
Ao, W., Gaudet, J., Kent, W. J., Muttumu, S., & Mango, S. E. (2004) · Science · doi:10.1126/science.1102216 · Cited by 194 · see citing works
Self-citationProceedings of the Eighth International Conference on Intelligent Systems for Molecular Biology
Bourne, P. E., Gribskov, M., Altman, R. B., et al. (2000) · Medical Entomology and Zoology · Cited by 133 · see citing works
Proceedings Of The Third International Conference On Intelligent Systems For Molecular Biology
Brenner, S. E., & Hubbard, T. (1995) · National Conference on Artificial Intelligence · Cited by 82 · see citing works
Rare Events and Conditional Events on Random Strings
Régnier, M., & Denise, A. (2004) · Discrete Mathematics & Theoretical Computer Science · doi:10.46298/dmtcs.310 · Cited by 57 · see citing works
Self-citationA Biologist's View of the Drosophila Genome Annotation Assessment ProjectPreprint
Ashburner, M. (2000) · Genome Research · doi:10.1101/gr.10.4.391 · Cited by 56 · see citing works
Performance comparison of algorithms for finding transcription factor binding sites
Sinha, S., & Tompa, M. (2003) · IEEE Symposium on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe.2003.1188949 · Cited by 23 · see citing works
Self-citationFavorov, A. V., & Gory, V. (2004) · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- Altman, R. et al.). 269-278 (AAAI Press, Menlo Park, CA, 2000).
- Bourbakis, N.G.). 214-220 (IEEE Computer Society, New York, 2003).
Cited by
Sorted by citations
Quantifying similarity between motifs
Gupta, S., Stamatoyannopoulos, J. A., Bailey, T. L., & Noble, W. S. (2007) · Genome Biology · doi:10.1186/gb-2007-8-2-r24 · Cited by 2,058 · see citing works
Self-citationTranscriptional control of human p53-regulated genes
Riley, T., Sontag, E., Chen, P., & Levine, A. (2008) · Nature Reviews Molecular Cell Biology · doi:10.1038/nrm2395 · Cited by 1,921 · see citing works
The evolution of gene regulation by transcription factors and microRNAs
Chen, K., & Rajewsky, N. (2007) · Nature Reviews Genetics · doi:10.1038/nrg1990 · Cited by 1,547 · see citing works
iRegulon: From a Gene List to a Gene Regulatory Network Using Large Motif and Track Collections
Janky, R., Verfaillie, A., Imrichová, H., Van de Sande, B., Standaert, L., Christiaens, V., Hulselmans, G., Herten, K., Naval Sanchez, M., Potier, D., et al. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003731 · Cited by 900 · see citing works
“Most of these methods performed well on yeast or bacterial promoter sequences, but they showed limited performances when applied to mouse or human [96].”
MethodTranscriptional Regulatory Elements in the Human Genome
Maston, G. A., Evans, S. K., & Green, M. R. (2006) · Annual Review of Genomics and Human Genetics · doi:10.1146/annurev.genom.7.080505.115623 · Cited by 937 · see citing works
Discovering motifs in ranked lists of DNA sequences
Eden, E., Lipson, D., Yogev, S., & Yakhini, Z. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030039 · Cited by 696 · see citing works
High-Resolution Mapping of Expression-QTLs Yields Insight into Human Gene Regulation
Veyrieras, J.-B., Kudaravalli, S., Kim, S. Y., Dermitzakis, E. T., Gilad, Y., Stephens, M., & Pritchard, J. K. (2008) · PLoS Genetics · doi:10.1371/journal.pgen.1000214 · Cited by 536 · see citing works
Absence of a simple code: how transcription factors read the genome
Slattery, M., Zhou, T., Yang, L., Dantas Machado, A. C., Gordân, R., & Rohs, R. (2014) · Trends in Biochemical Sciences · doi:10.1016/j.tibs.2014.07.002 · Cited by 537 · see citing works
Pique-Regi, R., Degner, J. F., Pai, A. A., Gaffney, D. J., Gilad, Y., & Pritchard, J. K. (2011) · Genome Research · doi:10.1101/gr.112623.110 · Cited by 521 · see citing works
Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders
Parikshak, N. N., Gandal, M. J., & Geschwind, D. H. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3934 · Cited by 430 · see citing works
Machine learning: a powerful tool for gene function prediction in plants
Mahood, E. H., Kruse, L. H., & Moghe, G. D. (2020) · Applications in Plant Sciences · doi:10.1002/aps3.11376 · Cited by 153 · see citing works
Brown, A. J., Snapkov, I., Akbar, R., Pavlović, M., Miho, E., Sandve, G. K., & Greiff, V. (2019) · Molecular Systems Design & Engineering · doi:10.1039/c9me00071b · Cited by 59 · see citing works
CompleteMOTIFs: DNA motif discovery platform for transcription factor binding experiments
Kuttippurathu, L., Hsing, M., Liu, Y., Schmidt, B., Maskell, D. L., Lee, K., He, A., Pu, W. T., & Kong, S. W. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btq707 · Cited by 48 · see citing works
Gamache, J., Gingerich, D., Shwab, E. K., Barrera, J., Garrett, M. E., Hume, C., Crawford, G. E., Ashley-Koch, A. E., & Chiba-Falek, O. (2023) · Cell & Bioscience · doi:10.1186/s13578-023-01120-5 · Cited by 20 · see citing works
Paredes, U. M., Quinn, J. P., & D'Souza, U. M. (2013) · Genes, Brain and Behavior · doi:10.1111/j.1601-183x.2012.00857.x · Cited by 10 · see citing works
An improved map of conserved regulatory sites for Saccharomyces cerevisiae
MacIsaac, K. D., Wang, T., Gordon, D. B., Gifford, D. K., Stormo, G. D., & Fraenkel, E. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-113 · Cited by 9 · see citing works
Interpreting noncoding genetic variation in complex traits and human disease
Ward, L. D., & Kellis, M. (2012) · Nature Biotechnology · doi:10.1038/nbt.2422 · Cited by 6 · see citing works
Data Integration Issues and Opportunities in Biological XML Data Management
Mesiti, M., Ruiz, E. J., Sanz, I., Llavori, R. B., Valentini, G., Perlasca, P., & Manset, D. (2010) · Open and Novel Issues in XML Database Applications · doi:10.4018/978-1-60566-308-1.ch012 · Cited by 4 · see citing works
- Authors
- Tompa, M., Li, N., Bailey, T. L., Church, G. M., De Moor, B., Eskin, E., Favorov, A. V., Frith, M. C., Fu, Y., Kent, W. J., Makeev, V. J., Mironov, A. A., Noble, W. S., Pavesi, G., Pesole, G., Régnier, M., Simonis, N., Sinha, S., Thijs, G., van Helden, J., Vandenbogaert, M., Weng, Z., Workman, C., Ye, C., & Zhu, Z.
- Affiliations
- University of Washington (US)
- The University of Queensland (AU)
- Harvard University (US)
- University of California San Diego (US)
- Engelhardt Institute of Molecular Biology (RU)
- Boston University (US)
- University of California, Santa Cruz (US)
- Lomonosov Moscow State University (RU)
- University of Milan (IT)
- Institut national de recherche en sciences et technologies du numérique (FR)
- Université Libre de Bruxelles (BE)
- Rockefeller University (US)
- KU Leuven (BE)
- Corresponding author
- Tompa, Martin
- ORCID
- Venue
- Nature Biotechnology
- Published
- 2005-01
- Volume
- 23
- Issue
- 1
- Pages
- 137-144
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- OpenAlex
- W2141408320
- Cited by
- 1,268
- References
- 25
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 12 · 2023: 11 · 2022: 14 · 2021: 17 · 2020: 27 · 2019: 24 · 2018: 31 · 2017: 27 · 2016: 49 · 2015: 48 · 2014: 43 · 2013: 79 · 2012: 90
- FWCI
- 53.08
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Nature Biotechnology (publishedVersion)
- PubMed (publishedVersion)
- Lirias (submittedVersion, other-oa)
- HAL AMU (submittedVersion)
- Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) (submittedVersion)
- CINECA IRIS Institutional Research Information System (University of Bari Aldo Moro) (submittedVersion)
- Funders
- National Science Foundation (0218798, 0116574)
- U.S. Department of Energy
- Cancer Research Institute
- W. M. Keck Foundation
- Ludwig Institute for Cancer Research
- European Commission
- Fédération Wallonie-Bruxelles
- Vlaamse regering
- National Institutes of Health
- Russian Academy of Sciences
- National Human Genome Research Institute (R01 HG 02602, R01HG003110, 1R01HG03110, R01HG002602)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Chromatin Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Keywords
- transcription factor binding sites, regulatory element prediction, computational tools, benchmarking, benchmark dataset, gene expression regulation
- MeSH terms
- Animals, Binding Sites, Drosophila, Fungal Proteins, Humans, Software, Transcription, Genetic, Reproducibility of Results, Gene Expression, Computational Biology, Internet, Amino Acid Motifs, Databases, Protein, Mice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
