Cited by 651Machine Learning in Materials ScienceAdvanced Chemical Physics StudiesComputational Drug Discovery Methods

Prediction Errors of Molecular Machine Learning Models Lower than Hybrid DFT Error

Open access
Faber, Felix A., Hutchison, Luke, Huang, Bing, Gilmer, Justin, Schoenholz, Samuel S., Dahl, George E., Vinyals, Oriol, Kearnes, Steven +2 more·2017·Journal of Chemical Theory and Computation

Abstract

We investigate the impact of choosing regressors and molecular representations for the construction of fast machine learning (ML) models of 13 electronic ground-state properties of organic molecules. The performance of each regressor/representation/property combination is assessed using learning curves which report out-of-sample errors as a function of training set size with up to ∼118k distinct molecules. Molecular structures and properties at the hybrid density functional theory (DFT) level of theory come from the QM9 database [ Ramakrishnan et al. Sci. Data 2014 , 1 , 140022 ] and include enthalpies and free energies of atomization, HOMO/LUMO energies and gap, dipole moment, polarizability, zero point vibrational energy, heat capacity, and the highest fundamental vibrational frequency. Various molecular representations have been studied (Coulomb matrix, bag of bonds, BAML and ECFP4, molecular graphs (MG)), as well as newly developed distribution based variants including histograms of distances (HD), angles (HDA/MARAD), and dihedrals (HDAD). Regressors include linear models (Bayesian ridge regression (BR) and linear regression with elastic net regularization (EN)), random forest (RF), kernel ridge regression (KRR), and two types of neural networks, graph convolutions (GC) and gated graph networks (GG). Out-of sample errors are strongly dependent on the choice of representation and regressor and molecular property. Electronic properties are typically best accounted for by MG and GC, while energetic properties are better described by HDAD and KRR. The specific combinations with the lowest out-of-sample errors in the ∼118k training set size limit are (free) energies and enthalpies of atomization (HDAD/KRR), HOMO/LUMO eigenvalue and gap (MG/GC), dipole moment (MG/GC), static polarizability (MG/GG), zero point vibrational energy (HDAD/KRR), heat capacity at room temperature (HDAD/KRR), and highest fundamental vibrational frequency (BAML/RF). We present numerical evidence that ML model predictions deviate from DFT (B3LYP) less than DFT (B3LYP) deviates from experiment for all properties. Furthermore, out-of-sample prediction errors with respect to hybrid DFT reference are on par with, or close to, chemical accuracy. The results suggest that ML models could be more accurate than hybrid DFT if explicitly electron correlated quantum (or experimental) data were available.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1021/acs.jctc.7b00577

Cites

62 of 62 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • CNNs are better at predicting quantum chemical energies.

    “For example, CNN is better at predicting quantum chemical energies.72”

    From QSAR without borders (2020) · cites this paper as [72]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Faber, F. A., Hutchison, L., Huang, B., Gilmer, J., Schoenholz, S. S., Dahl, G. E., Vinyals, O., Kearnes, S., Riley, P. F., & von Lilienfeld, O. A.
Affiliations
  • Google (United States) (US)
  • University of Basel (CH)
  • Google (United Kingdom) (GB)
  • Institute of Physical Chemistry and National Center for Computational Design and Discovery of Novel Materials, Department of Chemistry, University of Basel, Klingelbergstrasse 80, CH-4056 Basel, Switzerland
Venue
Journal of Chemical Theory and Computation
Published
2017-10-10
Volume
13
Issue
11
Pages
5255-5264
ISSN
  • 1549-9618 (ISSN-L)
  • 1549-9626
OpenAlex
W2753962198
Cited by
651
References
62
Citations by year
2026: 28 · 2025: 58 · 2024: 61 · 2023: 67 · 2022: 65 · 2021: 89 · 2020: 122 · 2019: 72 · 2018: 79 · 2017: 10
FWCI
32.95
Type
journal article
Language
en
Open access
Licence
other-oa
Locations
  • Journal of Chemical Theory and Computation (publishedVersion)
  • PubMed (publishedVersion)
  • edoc (University of Basel) (acceptedVersion, other-oa)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • edoc (University of Basel)
Funders
  • Schweizerischer Nationalfonds zur F?rderung der Wissenschaftlichen Forschung (310030_160067, PP00P2_138932)
  • Air Force Office of Scientific Research (FA9550-15-1-0026)
  • Google
  • Forschungsfonds, Universit?t Basel
Topics
  • Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
  • Physical Sciences › Physics and Astronomy › Atomic and Molecular Physics, and Optics › Advanced Chemical Physics Studies
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
Keywords
hybrid density functional theory, QM9, molecular representation, Coulomb matrix, ECFP4, molecular graphs, elastic net, kernel ridge regression, graph convolutional networks, random forest, enthalpy of atomization, HOMO-LUMO gap, dipole moment, polarizability, vibration characteristics, chemical accuracy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review