CoChat
Sign Up
Cited by 365Computational Drug Discovery MethodsMachine Learning in Materials Sciencevaccines and immunoinformatics approaches

Performance of machine-learning scoring functions in structure-based virtual screening.

Open access
Wójcikowski, Maciej, Ballester, Pedro J., Siedlecki, Pawel·2017·Scientific Reports
DOI 10.1038/srep46710PMC PMC5404222
Open this paper in CoChat — free accountOpen PDF

Abstract

Classical scoring functions have reached a plateau in their performance in virtual screening and binding affinity prediction. Recently, machine-learning scoring functions trained on protein-ligand complexes have shown great promise in small tailored studies. They have also raised controversy, specifically concerning model overfitting and applicability to novel targets. Here we provide a new ready-to-use scoring function (RF-Score-VS) trained on 15 426 active and 893 897 inactive molecules docked to a set of 102 targets. We use the full DUD-E data sets along with three docking tools, five classical and three machine-learning scoring functions for model building and performance assessment. Our results show RF-Score-VS can substantially improve virtual screening performance: RF-Score-VS top 1% provides 55.6% hit rate, whereas that of Vina only 16.2% (for smaller percent the difference is even more encouraging: RF-Score-VS top 0.1% achieves 88.6% hit rate for 27.5% using Vina). In addition, RF-Score-VS provides much better prediction of measured binding affinity than Vina (Pearson correlation of 0.56 and -0.18, respectively). Lastly, we test RF-Score-VS on an independent test set from the DEKOIS benchmark and observed comparable results. We provide full data sets to facilitate further research in this area (http://github.com/oddt/rfscorevs) as well as ready-to-use RF-Score-VS (http://github.com/oddt/rfscorevs_binary).

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/srep46710

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In Wójcikowski et al. (2017), fivefold cross-validations using the DUD-E dataset considered horizontal, vertical, and per-target splitting strategies.

    “For example, in the recent work of Wójcikowski et al. (2017), they performed fivefold cross-validations using the DUD-E dataset. Three distinct splitting strategies were considered: horizontal, vertical, and per-target.”

    From Empirical scoring functions for structure-based virtual screening: applications, critical aspects, and challenges (2018) · cites this paper as Wójcikowski et al., 2017

  • In the per-target split, training and test are performed for each protein target, simulating the construction and validation of target-specific scoring functions.

    “in the per-target split, the training and test are performed for each protein target (i.e., 102 unique machine-learning models relative to the 102 DUD-E targets), simulating the construction and validation of target-specific scoring functions.”

    From Empirical scoring functions for structure-based virtual screening: applications, critical aspects, and challenges (2018) · cites this paper as Wójcikowski et al., 2017

  • Tailored scoring functions are more beneficial for protein targets with fewer active compounds than for those containing more actives, where target-specific scoring functions exhibit similar performance to the generic model.

    “As an intriguing result, they found that tailored scoring functions are more beneficial for the protein targets with less active compounds than the others containing more actives, where the target-specific scoring functions exhibit similar performances to the generic model.”

    From Empirical scoring functions for structure-based virtual screening: applications, critical aspects, and challenges (2018) · cites this paper as Wójcikowski et al., 2017

  • Target-specific scoring functions trained with RF only performed slightly better than generic models, with two-thirds of them increasing EF1% less than 10%.

    “Still, in the recent work of Wójcikowski et al. (2017), the target-specific scoring functions trained with RF only performed slightly better than generic models, with two-third of them increasing the EF1% less than 10%.”

    From Empirical scoring functions for structure-based virtual screening: applications, critical aspects, and challenges (2018) · cites this paper as Wójcikowski et al., 2017

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Refaeilzadeh P., Tang L. & Liu H. In Encyclopedia of Database Systems (eds Liu L. & Tamer Özsu M.) 532– 538 (Springer: US, 2009).

Cited by

Sorted by citations

Related

Authors
Wójcikowski, M., Ballester, P. J., & Siedlecki, P.
Affiliations
  • Institute of Biochemistry and Biophysics, Polish Academy of Sciences (PL)
  • Centre National de la Recherche Scientifique (FR)
Corresponding author
  • Ballester, Pedro J.
  • Siedlecki, Pawel
ORCID
Venue
Scientific Reports
Publisher
Springer Science and Business Media LLC
Published
2017-04-25
Volume
7
Issue
1
Pages
46710-46710
ISSN
2045-2322 (ISSN-L)
DOI
10.1038/srep46710
PMC
PMC5404222
OpenAlex
W2608559058
Cited by
364
References
50
Citations by year
2026: 26 · 2025: 35 · 2024: 56 · 2023: 44 · 2022: 38 · 2021: 66 · 2020: 35 · 2019: 41 · 2018: 16 · 2017: 7
FWCI
20.32
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • ReIn UW (University of Warsaw) (submittedVersion, cc-by)
  • HAL (submittedVersion, other-oa)
Topics
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
  • Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › vaccines and immunoinformatics approaches
Keywords
machine learning scoring functions, structure-based virtual screening, protein-ligand docking, binding affinity prediction, overfitting, DUD-E, Vina, hit rate

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review