CoChat
Sign Up
Cited by 106Genomics and Chromatin DynamicsBioinformatics and Genomic NetworksGenetic Associations and Epidemiology

A curated benchmark of enhancer-gene interactions for evaluating enhancer-target gene prediction methods

Open access
Moore, Jill E., Pratt, Henry E., Purcaro, Michael J., Weng, Zhiping·2020·Genome Biology
DOI 10.1186/s13059-019-1924-8PMC PMC6977301

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Many genome-wide collections of candidate cis-regulatory elements (cCREs) have been defined using genomic and epigenomic data, but it remains a major challenge to connect these elements to their target genes. Results: To facilitate the development of computational methods for predicting target genes, we develop a Benchmark of candidate Enhancer-Gene Interactions (BENGI) by integrating the recently developed Registry of cCREs with experimentally derived genomic interactions. We use BENGI to test several published computational methods for linking enhancers with genes, including signal correlation and the TargetFinder and PEP supervised learning methods. We find that while TargetFinder is the best-performing method, it is only modestly better than a baseline distance method for most benchmark datasets when trained and tested with the same cell type and that TargetFinder often does not outperform the distance method when applied across cell types. Conclusions: Our results suggest that current computational methods need to be improved and that BENGI presents a useful framework for method development and testing.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13059-019-1924-8

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • STAT3 activation potentiates IL-6 signaling by increasing receptor and ligand levels.

    “STAT3 activation potentiates IL-6 signaling by increasing receptor and ligand levels”

    From Dynamic IL-6R/STAT3 signaling leads to heterogeneity of metabolic phenotype in pancreatic ductal adenocarcinoma cells (2024) · cites this paper as [36]

    The Results section discusses enhancer–gene interaction datasets and gene expression, not IL-6 signaling. Searches for STAT3 activation and for STAT3, IL-6, receptor, and ligand returned no passage addressing the claimed mechanism. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 41 · Cited by 106

Research works printed but not linked: 32

  1. [1]Ernst J Kheradpour P Mikkelsen TS Shoresh N Ward LD Epstein CB Mapping and analysis of chromatin state dynamics in nine human cell types Nature. 2011 473 43 49 10.1038/nature09906 21441907 PMC3088773 · DOI · Paper page
  2. [2]ENCODE Project Consortium Bernstein BE Birney E Dunham I Green ED Gunter C An integrated encyclopedia of DNA elements in the human genome Nature 2012 489 57 74 10.1038/nature11247 22955616 PMC3439153 · DOI · Paper page
  3. [3]Hoffman MM Buske OJ Wang J Weng Z Bilmes JA Noble WS Unsupervised pattern discovery in human chromatin structure through genomic segmentation Nat Methods 2012 9 473 476 10.1038/nmeth.1937 22426492 PMC3340533 · DOI
  4. [4]Rajagopal N Xie W Li Y Wagner U Wang W Stamatoyannopoulos J RFECS: A Random-Forest Based Algorithm for Enhancer Identification from Chromatin State. Singh M, editor PLoS Comput Biol Public Libr Sci 2013 9 e1002968 10.1371/journal.pcbi.1002968 PMC3597546 23526891 · DOI
  5. [6]He Y Gorkin DU Dickel DE Nery JR Castanon RG Lee AY Improved regulatory element prediction based on tissue-specific local epigenomic signatures Proc Natl Acad Sci U S A 2017 114 E1633 E1640 10.1073/pnas.1618353114 28193886 PMC5338528 · DOI
  6. [7]Wang D Liu S Warrell J Won H Shi X Navarro FCP Comprehensive functional genomic resource and integrative model for the human brain Science 2018 362 eaat8464 10.1126/science.aat8464 30545857 PMC6413328 · DOI · Paper page
  7. [9]Schaub MA Boyle AP Kundaje A Batzoglou S Snyder M Linking disease associations with regulatory information in the human genome Genome Res 2012 22 1748 1759 10.1101/gr.136127.111 22955986 PMC3431491 · DOI
  8. [10]Farh KK-H Marson A Zhu J Kleinewietfeld M Housley WJ Beik S Genetic and epigenetic fine mapping of causal autoimmune disease variants Nature. 2014 518 337 343 10.1038/nature13835 25363779 PMC4336207 · DOI · Paper page
  9. [11]Finucane HK Bulik-Sullivan B Gusev A Trynka G Reshef Y Loh P-R Partitioning heritability by functional annotation using genome-wide association summary statistics Nature Genet 2015 47 1228 1235 10.1038/ng.3404 26414678 PMC4626285 · DOI · Paper page
  10. [13]Lieberman-Aiden E van Berkum NL Williams L Imakaev M Ragoczy T Telling A Comprehensive mapping of long-range interactions reveals folding principles of the human genome Sci Am Assoc Adv Sci 2009 326 289 293 PMC2858594 19815776
  11. [14]Rao SSP Huntley MH Durand NC Stamenova EK Bochkov ID Robinson JT A 3D map of the human genome at kilobase resolution reveals principles of chromatin looping Cell. 2014 159 1665 1680 10.1016/j.cell.2014.11.021 25497547 PMC5635824 · DOI · Paper page
  12. [15]Li G Ruan X Auerbach RK Sandhu KS Zheng M Wang P Extensive promoter-centered chromatin interactions provide a topological basis for transcription regulation Cell. 2012 148 84 98 10.1016/j.cell.2011.12.014 22265404 PMC3339270 · DOI · Paper page
  13. [16]Tang Z Luo OJ Li X Zheng M Zhu JJ Szalaj P CTCF-mediated human 3D genome architecture reveals chromatin topology for transcription Cell. 2015 163 1611 1627 10.1016/j.cell.2015.11.024 26686651 PMC4734140 · DOI · Paper page
  14. [17]Mifsud B Tavares-Cadete F Young AN Sugar R Schoenfelder S Ferreira L Mapping long-range promoter contacts in human cells with high-resolution capture hi-C Nat Genet 2015 47 598 606 10.1038/ng.3286 25938943 · DOI · Paper page
  15. [18]Lappalainen T Sammeth M Friedländer MR PAC 'tH Monlong J Rivas MA Transcriptome and genome sequencing uncovers functional variation in humans Nature. 2013 501 506 511 10.1038/nature12531 24037378 PMC3918453 · DOI · Paper page
  16. [19]GTEx Consortium, Laboratory, Data Analysis &Coordinating Center (LDACC)—Analysis Working Group, Statistical Methods groups—Analysis Working Group, Enhancing GTEx (eGTEx) groups, NIH Common Fund, NIH/NCI Genetic effects on gene expression across human tissues Nature 2017 550 204 213 10.1038/nature24277 29022597 PMC5776756 · DOI · Paper page
  17. [20]Gasperini M Hill AJ McFaline-Figueroa JL Martin B Kim S Zhang MD A Genome-wide Framework for Mapping Gene Regulation via Cellular Genetic Screens Cell 2019 176 377 390.e19 10.1016/j.cell.2018.11.029 30612741 PMC6690346 · DOI · Paper page
  18. [22]Thurman RE Rynes E Humbert R Vierstra J Maurano MT Haugen E The accessible chromatin landscape of the human genome Nature 2012 489 75 82 10.1038/nature11232 22955617 PMC3721348 · DOI · Paper page
  19. [23]Sheffield NC Thurman RE Song L Safi A Stamatoyannopoulos JA Lenhard B Patterns of regulatory activity across diverse human cell types predict tissue identity, transcription factor binding, and long-range interactions Genome Res 2013 23 777 788 10.1101/gr.152140.112 23482648 PMC3638134 · DOI
  20. [24]He B Chen C Teng L Tan K Global view of enhancer-promoter interactome in human cells Proc Natl Acad Sci USA 2014 111 E2191 E2199 10.1073/pnas.1320308111 24821768 PMC4040567 · DOI
  21. [25]Roy S Siahpirani AF Chasman D Knaack S Ay F Stewart R A predictive modeling approach for cell line-specific long-range regulatory interactions Nucleic Acids Res 2015 43 8694 8712 10.1093/nar/gkv865 26338778 PMC4605315 · DOI
  22. [26]Zhao C Li X Hu H PETModule: a motif module based approach for enhancer target gene prediction Sci Rep 2016 6 30043 10.1038/srep30043 27436110 PMC4951774 · DOI
  23. [27]Whalen S Truty RM Pollard KS Enhancer-promoter interactions are encoded by complex genomic signatures on looping chromatin Nat Genet Nat Res 2016 48 488 496 10.1038/ng.3539 PMC4910881 27064255 · DOI · Paper page
  24. [28]Yang Y Zhang R Singh S Ma J Exploiting sequence-based features for predicting enhancer–promoter interactions Bioinformatics 2017 33 i252 i260 10.1093/bioinformatics/btx257 28881991 PMC5870728 · DOI
  25. [29]Xi W Beer MA Local epigenomic state cannot discriminate interacting and non-interacting enhancer-promoter pairs with high accuracy. Noble WS, editor PLoS Comput Biol 2018 14 e1006625 10.1371/journal.pcbi.1006625 30562350 PMC6298642 · DOI · Paper page
  26. [30]Cao F Fullwood MJ Inflated performance measures in enhancer-promoter interaction-prediction methods Nature Genetics 2019 326 289 1198 31332378
  27. [31]Fishilevich S Nudel R Rappaport N Hadar R Plaschkes I Iny Stein T GeneHancer: genome-wide integration of enhancers and target genes in GeneCards Database (Oxford) 2017 2017 1217 10.1093/database/bax028 PMC5467550 28605766 · DOI
  28. [32]Schreiber J, Singh R, Bilmes J, bioRxiv WN, 2019. A pitfall for machine learning methods aiming to predict across cell types. biorxiv.org. Accessed 1 June 2019. PMC7678316 33213499
  29. [33]Forcato M Nicoletti C Pal K Livi CM Ferrari F Bicciato S Comparison of computational methods for hi-C data analysis Nat Methods 2017 14 679 685 10.1038/nmeth.4325 28604721 PMC5493985 · DOI
  30. [35]Moore JE, Pratt HE, Purcaro MJ, Weng Z. A curated benchmark of enhancer-gene interactions for evaluating enhancer-target gene prediction methods. Github. 2019. https://github.com/weng-lab/BENGI. Accessed 1 Dec 2019. PMC6977301 31969180
  31. [36]Grant CE Bailey TL Noble WS FIMO: scanning for occurrences of a given motif Bioinformatics. 2011 27 1017 1018 10.1093/bioinformatics/btr064 21330290 PMC3065696 · DOI · Paper page
  32. [37]Kulakovskiy IV Vorontsov IE Yevshin IS Sharipov RN Fedorova AD Rumynskiy EI HOCOMOCO: towards a complete collection of transcription factor binding models for human and mouse via large-scale ChIP-Seq analysis Nucleic Acids Res 2018 46 D252 D259 10.1093/nar/gkx1106 29140464 PMC5753240 · DOI

Authors
Moore, J. E., Pratt, H. E., Purcaro, M. J., & Weng, Z.
Affiliations
University of Massachusetts Chan Medical School (US)
Corresponding author
Weng, Zhiping
ORCID
Venue
Genome Biology
Publisher
Springer Science and Business Media LLC
Published
2020-01-22
Volume
21
Issue
1
Pages
17-17
ISSN
  • 1465-6906 (ISSN-L)
  • 1474-760X
DOI
10.1186/s13059-019-1924-8
PMC
PMC6977301
OpenAlex
W3002247544
Cited by
106
References
41
Citations by year
2026: 7 · 2025: 10 · 2024: 9 · 2023: 26 · 2022: 25 · 2021: 19 · 2020: 9 · 2012: 1
FWCI
6.57
Type
journal article
Language
en
Open access
Licence
Locations
  • Genome biology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Human Genome Research Institute (1U24HG009446-01, 5U41HG007000)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Chromatin Dynamics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
Keywords
enhancer-gene interactions, cis-regulatory elements, benchmark dataset, PEP, signal correlation, cell type specificity, epigenomic data
MeSH terms
Machine Learning, Enhancer Elements, Genetic, Gene Expression Regulation, Benchmarking, Genomics, Data Curation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review