Cited by 125Genomics and Chromatin DynamicsGenomics and Phylogenetic StudiesMachine Learning in Bioinformatics

DEEP: a general computational framework for predicting enhancers

Open access
Kleftogiannis, Dimitrios, Kalnis, Panos, Bajic, Vladimir B.·2015·Nucleic Acids Research

Abstract

Transcription regulation in multicellular eukaryotes is orchestrated by a number of DNA functional elements located at gene regulatory regions. Some regulatory regions (e.g. enhancers) are located far away from the gene they affect. Identification of distal regulatory elements is a challenge for the bioinformatics research. Although existing methodologies increased the number of computationally predicted enhancers, performance inconsistency of computational models across different cell-lines, class imbalance within the learning sets and ad hoc rules for selecting enhancer candidates for supervised learning, are some key questions that require further examination. In this study we developed DEEP, a novel ensemble prediction framework. DEEP integrates three components with diverse characteristics that streamline the analysis of enhancer's properties in a great variety of cellular conditions. In our method we train many individual classification models that we combine to classify DNA regions as enhancers or non-enhancers. DEEP uses features derived from histone modification marks or attributes coming from sequence characteristics. Experimental results indicate that DEEP performs better than four state-of-the-art methods on the ENCODE data. We report the first computational enhancer prediction results on FANTOM5 data where DEEP achieves 90.2% accuracy and 90% geometric mean (GM) of specificity and sensitivity across 36 different tissues. We further present results derived using in vivo-derived enhancer data from VISTA database. DEEP-VISTA, when tested on an independent test set, achieved GM of 80.1% and accuracy of 89.64%. DEEP framework is publicly available at http://cbrc.kaust.edu.sa/deep/.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/nar/gku1058

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Massively parallel reporter assays coupled with machine learning have proven useful in addressing similar voids by producing quantitative biological insight that would be difficult to achieve through traditional approaches.

    “Massively parallel reporter assays (MPRAs) – methods that assess thousands to millions of sequence variants in a single experiment – coupled with machine learning have proven useful in addressing similar voids by producing quantitative biological insight that would be difficult to achieve through traditional approaches3–9.”

    From Human 5′ UTR design and variant effect prediction from a massively parallel translation assay (2019) · cites this paper as [6]

    The Introduction discusses machine learning applied to histone-mark and other high-throughput data, while the Materials and Methods describe DEEP’s use of ChIP-Seq data or DNA sequences. Searches for massively parallel reporter assays and reporter assays coupled with machine learning found no passage addressing that combination or the claimed quantitative insight. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 37

  1. [1]Lander E.S. Linton L.M. Birren B. Nusbaum C. Zody M.C. Baldwin J. Devon K. Dewar K. Doyle M. FitzHugh W. Initial sequencing and analysis of the human genome Nature 2001 409 860 921 11237011
  2. [2]Venter J.C. Adams M.D. Myers E.W. Li P.W. Mural R.J. Sutton G.G. Smith H.O. Yandell M. Evans C.A. Holt R.A. The sequence of the human genome Science 2001 291 1304 1351 11181995
  3. [4]Heintzman N.D. Ren B. Finding distal regulatory elements in the human genome Curr. Opin. Genet. Dev. 2009 19 541 549 19854636 PMC3321269
  4. [5]Lee T.I. Young R.A. Transcription of eukaryotic protein-coding genes Ann. Rev. Genet. 2000 34 77 137 11092823
  5. [6]Plank J.L. Dean A. Enhancer function: mechanistic and genome-wide insights come together Mol. Cell 2014 55 5 14 24996062 PMC8441652
  6. [7]Glass C.K. Rosenfeld M.G. The coregulator exchange in transcriptional functions of nuclear receptors Genes Dev. 2000 14 121 141 10652267
  7. [8]West A.G. Gaszner M. Felsenfeld G. Insulators: many functions, many mechanisms Genes Dev. 2002 16 271 288 11825869
  8. [9]Skipper M. Dhand R. Campbell P. Presenting ENCODE Nature 2012 489 45 22955612
  9. [10]Bernstein B.E. Stamatoyannopoulos J.A. Costello J.F. Ren B. Milosavljevic A. Meissner A. Kellis M. Marra M.A. Beaudet A.L. Ecker J.R. The NIH roadmap epigenomics mapping consortium Nat. Biotechnol. 2010 28 1045 1048 20944595 PMC3607281
  10. [11]The FANTOM Consortium and the RIKEN PMI and CLST (DGT) A promoter-level mammalian expression atlas Nature 2014 507 462 470 24670764 PMC4529748
  11. [12]Ernst J. Kheradpour P. Mikkelsen T.S. Shoresh N. Ward L.D. Epstein C.B. Zhang X. Wang L. Issner R. Coyne M. Mapping and analysis of chromatin state dynamics in nine human cell types Nature 2011 473 43 49 21441907 PMC3088773
  12. [13]Visel A. Blow M.J. Li Z. Zhang T. Akiyama J.A. Holt A. Plajzer-Frick I. Shoukry M. Wright C. Chen F. ChIP-seq accurately predicts tissue-specific activity of enhancers Nature 2009 457 854 858 19212405 PMC2745234
  13. [14]Visel A. Prabhakar S. Akiyama J.A. Shoukry M. Lewis K.D. Holt A. Plajzer-Frick I. Afzal V. Rubin E.M. Pennacchio L.A. Ultraconservation identifies a small subset of extremely constrained developmental enhancers Nat. Genet. 2008 40 158 160 18176564 PMC2647775
  14. [16]Boyle A.P. Song L. Lee B.K. London D. Keefe D. Birney E. Iyer V.R. Crawford G.E. Furey T.S. High-resolution genome-wide in vivo footprinting of diverse transcription factors in human cells Genome Res. 2011 21 456 464 21106903 PMC3044859
  15. [17]Matys V. Kel-Margoulis O.V. Fricke E. Liebich I. Land S. Barre-Dirrie A. Reuter I. Chekmenev D. Krull M. Hornischer K. TRANSFAC and its module TRANSCompel: transcriptional gene regulation in eukaryotes Nucleic Acids Res. 2006 34 D108 D110 16381825 PMC1347505
  16. [18]Bryne J.C. Valen E. Tang M.H. Marstrand T. Winther O. da Piedade I. Krogh A. Lenhard B. Sandelin A. JASPAR, the open access database of transcription factor-binding profiles: new content and tools in the 2008 update Nucleic Acids Res. 2008 36 D102 D106 18006571 PMC2238834
  17. [19]Kulakovskiy I.V. Medvedeva Y.A. Schaefer U. Kasianov A.S. Vorontsov I.E. Bajic V.B. Makeev V.J. HOCOMOCO: a comprehensive collection of human transcription factor binding sites models Nucleic Acids Res. 2013 41 D195 D202 23175603 PMC3531053
  18. [20]Ravasi T. Suzuki H. Cannistraci C.V. Katayama S. Bajic V.B. Tan K. Akalin A. Schmeier S. Kanamori-Katayama M. Bertin N. An atlas of combinatorial transcriptional regulation in mouse and man Cell 2010 140 744 752 20211142 PMC2836267
  19. [21]Ernst J. Kellis M. ChromHMM: automating chromatin-state discovery and characterization Nat. Methods 2012 9 215 216 22373907 PMC3577932
  20. [22]Hoffman M.M. Buske O.J. Wang J. Weng Z. Bilmes J.A. Noble W.S. Unsupervised pattern discovery in human chromatin structure through genomic segmentation Nat. Methods 2012 9 473 476 22426492 PMC3340533
  21. [23]Firpi H.A. Ucar D. Tan K. Discover regulatory DNA elements using chromatin signatures and artificial neural network Bioinformatics 2010 26 1579 1586 20453004 PMC2887052
  22. [24]Fernandez M. Miranda-Saavedra D. Genome-wide enhancer prediction from epigenetic signatures using genetic algorithm-optimized support vector machines Nucleic Acids Res. 2012 40 e77 22328731 PMC3378905
  23. [25]Rajagopal N. Xie W. Li Y. Wagner U. Wang W. Stamatoyannopoulos J. Ernst J. Kellis M. Ren B. RFECS: a random-forest based algorithm for enhancer identification from chromatin state PLoS Comput. Biol. 2013 9 e1002968 23526891 PMC3597546
  24. [26]Erwin G.D. Oksenberg N. Truty R.M. Kostka D. Murphy K.K. Ahituv N. Pollard K.S. Capra J.A. Integrating diverse datasets improves developmental enhancer prediction PLoS Comput. Biol. 2014 10 e1003677 24967590 PMC4072507
  25. [28]Visel A. Minovitsky S. Dubchak I. Pennacchio L.A. VISTA Enhancer Browser–a database of tissue-specific human enhancers Nucleic Acids Res. 2007 35 D88 D92 17130149 PMC1716724
  26. [29]Ernst J. Kellis M. Discovery and characterization of chromatin states for systematic annotation of the human genome Nat. Biotechnol. 2010 28 817 825 20657582 PMC2919626
  27. [30]Hoffman M.M. Ernst J. Wilder S.P. Kundaje A. Harris R.S. Libbrecht M. Giardine B. Ellenbogen P.M. Bilmes J.A. Birney E. Integrative annotation of chromatin elements from ENCODE data Nucleic Acids Res. 2013 41 827 841 23221638 PMC3553955
  28. [31]Blanchette M. Bataille A.R. Chen X. Poitras C. Laganiere J. Lefebvre C. Deblois G. Giguere V. Ferretti V. Bergeron D. Genome-wide computational prediction of transcriptional regulatory modules reveals new insights into human gene expression Genome Res. 2006 16 656 668 16606704 PMC1457048
  29. [33]Ernst J. Kellis M. Interplay between chromatin state, regulator binding, and regulatory motifs in six human cell types Genome Res. 2013 23 1142 1154 23595227 PMC3698507
  30. [34]Batuwita R. Palade V. Imbalanced Learning 2013 John Wiley & Sons, Inc. 83 99
  31. [35]Schölkopf B. Burges C.J. Smola A.J. Advances in Kernel Methods: Support Vector Learning 1999 Cambridge, MA MIT Press
  32. [36]Wu G. Chang E. Y. Adaptive feature-space conformal transformation for imbalanced data learning Proceedings of the Twentieth International Conference on Machine Learning 2003 20 2003
  33. [37]Akbani R. Kwek S. Japkowicz N. Boulicaut J-F Esposito F Giannotti F Pedreschi D Machine Learning: ECML 2004 2004 3201 Heidelberg, Berlin Springer 39 50
  34. [38]Ren B. Transcription: enhancers make non-coding RNA Nature 2010 465 173 174 20463730
  35. [39]Yang C. Bolotin E. Jiang T. Sladek F.M. Martinez E. Prevalence of the initiator over the TATA box in human and yeast genes and identification of DNA motifs enriched in human TATA-less core promoters Gene 2007 389 52 65 17123746 PMC1955227
  36. [40]Bajic V.B. Comparing the success of different prediction software in sequence analysis: a review Briefings Bioinformat. 2000 1 214 228 11465033
  37. [41]Pizzi C. Rastas P. Ukkonen E. Finding significant matches of position weight matrices in linear time IEEE/ACM Trans. Comput. Biol. Bioinformat. 2011 8 69 79 21071798

Cited by

Sorted by citations

Authors
Kleftogiannis, D., Kalnis, P., & Bajic, V. B.
Affiliations
King Abdullah University of Science and Technology (SA)
Venue
Nucleic Acids Research
Published
2014-11-05
Volume
43
Issue
1
Pages
e6-e6
ISSN
  • 0305-1048 (ISSN-L)
  • 1362-4962
OpenAlex
W2140240158
Cited by
125
References
4
Citations by year
2026: 3 · 2025: 3 · 2024: 6 · 2023: 8 · 2022: 17 · 2021: 15 · 2020: 13 · 2019: 12 · 2018: 18 · 2017: 9 · 2016: 14 · 2015: 7
FWCI
3.77
Type
journal article
Language
en
Open access
Locations
  • Nucleic Acids Research (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • King Abdullah University of Science and Technology Repository (King Abdullah University of Science and Technology) (submittedVersion, other-oa)
Funders
King Abdullah University of Science and Technology
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Chromatin Dynamics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
Keywords
enhancer prediction, distal regulatory elements, transcriptional regulation, ensemble learning, sequence features, cell line specificity, class imbalance, FANTOM5, ENCODE, DEEP
MeSH terms
Enhancer Elements, Genetic, HeLa Cells, Histones, Humans, Transcription Factors, Sequence Analysis, DNA, K562 Cells, Genomics, Chromatin Immunoprecipitation, Support Vector Machine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review