Machine-learning-assisted materials discovery using failed experiments
The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
Inorganic-organic hybrid materials such as organically templated metal oxides, metal-organic frameworks (MOFs) and organohalide perovskites have been studied for decades, and hydrothermal and (non-aqueous) solvothermal syntheses have produced thousands of new materials that collectively contain nearly all the metals in the periodic table. Nevertheless, the formation of these compounds is not fully understood, and development of new compounds relies primarily on exploratory syntheses. Simulation- and data-driven approaches (promoted by efforts such as the Materials Genome Initiative) provide an alternative to experimental trial-and-error. Three major strategies are: simulation-based predictions of physical properties (for example, charge mobility, photovoltaic properties, gas adsorption capacity or lithium-ion intercalation) to identify promising target candidates for synthetic efforts; determination of the structure-property relationship from large bodies of experimental data, enabled by integration with high-throughput synthesis and measurement tools; and clustering on the basis of similar crystallographic structure (for example, zeolite structure classification or gas adsorption properties). Here we demonstrate an alternative approach that uses machine-learning algorithms trained on reaction data to predict reaction outcomes for the crystallization of templated vanadium selenites. We used information on 'dark' reactions--failed or unsuccessful hydrothermal syntheses--collected from archived laboratory notebooks from our laboratory, and added physicochemical property descriptions to the raw notebook information using cheminformatics techniques. We used the resulting data to train a machine-learning model to predict reaction success. When carrying out hydrothermal synthesis experiments using previously untested, commercially available organic building blocks, our machine-learning model outperformed traditional human strategies, and successfully predicted conditions for new organically templated inorganic product formation with a success rate of 89 per cent. Inverting the machine-learning model reveals new hypotheses regarding the conditions for successful product formation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, in experimental data sets extracted from the literature (14,15,19), there can be positive bias for reporting only successful experiments (16, 107, 108), leading to challenges in classification tasks (17).”
From Machine Learning for the Discovery, Design, and Engineering of Materials. (2022) · cites this paper as [108]
In the citing paper
“ML has had recent success in predicting specific chemical, material, and optical properties without the need for expensive experimentation or brute-force simulation [105,299,490,653], including in aerosols [465,500].”
From Tackling climate change with machine learning (2023) · cites this paper as [653]
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Cortes, C., & Vapnik, V. (1995) · Machine-mediated learning · doi:10.1023/a:1022627411411 · Cited by 48,445
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · cited 1×
“The SVM was implemented in WEKA 3.7 32,33 ; this implementation included a built-in data-normalization step.”
MethodInfluentialThe Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · cited 2×
“All data were relabelled with the predicted outcomes of the SVM model and a C4.5 decision tree (implemented in WEKA 3.7) 32 was used to model those predicted outcomes 39 .”
MethodInfluentialThe Cambridge Structural Database: a quarter of a million crystal structures and rising
Allen, F. H. (2002) · Acta Crystallographica Section B Structural Science · doi:10.1107/s0108768102003890 · Cited by 9,898 · cited 2×
“Organically templated metal oxides using these diamines are essentially unknown, as indicated by their near absence from the Cambridge Structural Database 26 (see Methods).”
BackgroundText categorization with Support Vector Machines: Learning with many relevant features
Joachims, T. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0026683 · Cited by 7,999
Introduction to Metal–Organic Frameworks
Zhou, H.-C., Long, J. R., & Yaghi, O. M. (2012) · Chemical Reviews · doi:10.1021/cr300014x · Cited by 7,720 · cited 1×
“Inorganic–organic hybrid materials 1–3 such as organically templated metal oxides 1 , metal–organic frameworks (MOFs) 2 and organohalide perovskites 4 have been studied for decades, and hydrothermal and (non-aqueous) solvothermal syntheses have produced thousands of new materials that collectively…”
BackgroundMetal-halide perovskites for photovoltaic and light-emitting devices
Stranks, S. D., & Snaith, H. J. (2015) · Nature Nanotechnology · doi:10.1038/nnano.2015.90 · Cited by 3,392 · cited 1×
“Inorganic–organic hybrid materials 1–3 such as organically templated metal oxides 1 , metal–organic frameworks (MOFs) 2 and organohalide perovskites 4 have been studied for decades, and hydrothermal and (non-aqueous) solvothermal syntheses have produced thousands of new materials that collectively…”
BackgroundOpen-Framework Inorganic Materials
Cheetham, A. K., Férey, G., & Loiseau, T. (1999) · Angewandte Chemie International Edition · doi:10.1002/(sici)1521-3773(19991115)38:22<3268::aid-anie3268>3.0.co;2-u · Cited by 2,426 · cited 1×
“…templated metal oxides 1 , metal–organic frameworks (MOFs) 2 and organohalide perovskites 4 have been studied for decades, and hydrothermal and (non-aqueous) solvothermal syntheses have produced thousands of new materials that collectively contain nearly all the metals in the periodic table 5–9 .”
BackgroundCundy, C. S., & Cox, P. A. (2003) · Chemical Reviews · doi:10.1021/cr020060i · Cited by 2,126
Férey, G. (2001) · Chemistry of Materials · doi:10.1021/cm011070n · Cited by 659 · cited 1×
“Inorganic–organic hybrid materials 1–3 such as organically templated metal oxides 1 , metal–organic frameworks (MOFs) 2 and organohalide perovskites 4 have been studied for decades, and hydrothermal and (non-aqueous) solvothermal syntheses have produced thousands of new materials that collectively…”
BackgroundNew Stories of Zeolite Structures: Their Descriptions, Determinations, Predictions, and Evaluations
Li, Y., & Yu, J. (2014) · Chemical Reviews · doi:10.1021/cr500010r · Cited by 603 · cited 1×
“…relationship from large bodies of experimental data 16,17 , enabled by integration with high-throughput synthesis and measurement tools 18 ; and clustering on the basis of similar crystallographic structure (for example, zeolite structure classification 19,20 or gas adsorption properties 21 ).”
BackgroundHautier, G., Fischer, C. C., Jain, A., Mueller, T., & Ceder, G. (2010) · Chemistry of Materials · doi:10.1021/cm100795d · Cited by 580 · cited 1×
“…of physical properties (for example, charge mobility 11 , photovoltaic properties 12 , gas adsorption capacity 13 or lithium-ion intercalation 14 ) to identify promising target candidates for synthetic efforts 11,15 ; determination of the structure–property relationship from large bodies of…”
BackgroundAn Introduction To Chemoinformatics
Leach, A. R., & Gillet, V. J. (2007) · Springer eBooks · doi:10.1007/978-1-4020-6291-9 · Cited by 533
Data-Driven Review of Thermoelectric Materials: Performance and Resource Considerations
Gaultois, M. W., Sparks, T. D., Borg, C. K. H., Seshadri, R., Bonificio, W. D., & Clarke, D. R. (2013) · Chemistry of Materials · doi:10.1021/cm400893e · Cited by 500 · cited 1×
“…target candidates for synthetic efforts 11,15 ; determination of the structure–property relationship from large bodies of experimental data 16,17 , enabled by integration with high-throughput synthesis and measurement tools 18 ; and clustering on the basis of similar crystallographic…”
BackgroundHigh-throughput computational screening of metal–organic frameworks
Colón, Y. J., & Snurr, R. Q. (2014) · Chem. Soc. Rev. · doi:10.1039/c4cs00070f · Cited by 470 · cited 1×
“…are: simulation-based predictions of physical properties (for example, charge mobility 11 , photovoltaic properties 12 , gas adsorption capacity 13 or lithium-ion intercalation 14 ) to identify promising target candidates for synthetic efforts 11,15 ; determination of the structure–property…”
BackgroundHaushalter, R. C., & Mundi, L. A. (1992) · Chemistry of Materials · doi:10.1021/cm00019a012 · Cited by 437
Open-source platform to benchmark fingerprints for ligand-based virtual screening
Riniker, S., & Landrum, G. A. (2013) · Journal of Cheminformatics · doi:10.1186/1758-2946-5-26 · Cited by 359
Sokolov, A. N., Atahan-Evrenk, S., Mondal, R., Akkerman, H. B., Sánchez-Carrera, R. S., Granados-Focil, S., Schrier, J., Mannsfeld, S. C. B., Zoombelt, A. P., Bao, Z., et al. (2011) · Nature Communications · doi:10.1038/ncomms1451 · Cited by 356
Self-citationÜstün, B., Melssen, W. J., & Buydens, L. M. C. (2006) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2005.09.003 · Cited by 305 · cited 1×
“Specifically, a SVM 30 model with a universal Pearson VII function-based kernel 31 was trained on 3,955 labelled reactions previously performed by the laboratory.”
MethodFernandez, M., Boyd, P. G., Daff, T. D., Aghaji, M. Z., & Woo, T. K. (2014) · The Journal of Physical Chemistry Letters · doi:10.1021/jz501331m · Cited by 296
Materials Data Science: Current Status and Future Outlook
Kalidindi, S. R., & De Graef, M. (2015) · Annual Review of Materials Research · doi:10.1146/annurev-matsci-070214-020844 · Cited by 294 · cited 1×
“…target candidates for synthetic efforts 11,15 ; determination of the structure–property relationship from large bodies of experimental data 16,17 , enabled by integration with high-throughput synthesis and measurement tools 18 ; and clustering on the basis of similar crystallographic…”
BackgroundFérey, G. (1995) · Journal of Fluorine Chemistry · doi:10.1016/0022-1139(94)00406-6 · Cited by 265
Overview of Supervised Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7_2 · Cited by 259
Hachmann, J., Olivares-Amaya, R., Jinich, A., Appleton, A. L., Blood-Forsythe, M. A., Seress, L. R., Román-Salgado, C., Trepte, K., Atahan-Evrenk, S., Er, S., et al. (2014) · Energy Environ. Sci. · doi:10.1039/c3ee42756k · Cited by 247 · cited 1×
“Three major strategies are: simulation-based predictions of physical properties (for example, charge mobility 11 , photovoltaic properties 12 , gas adsorption capacity 13 or lithium-ion intercalation 14 ) to identify promising target candidates for synthetic efforts 11,15 ; determination of the…”
BackgroundExploration of a Simple Universal Route to the Myriad of Open-Framework Metal Phosphates
Rao, C. N. R., Natarajan, S., & Neeraj, S. (2000) · Journal of the American Chemical Society · doi:10.1021/ja993892f · Cited by 206 · cited 1×
“…templated metal oxides 1 , metal–organic frameworks (MOFs) 2 and organohalide perovskites 4 have been studied for decades, and hydrothermal and (non-aqueous) solvothermal syntheses have produced thousands of new materials that collectively contain nearly all the metals in the periodic table 5–9 .”
BackgroundOrganically-templated metal sulfates, selenites and selenates
Rao, C. N. R., Behera, J. N., & Dan, M. (2006) · Chemical Society Reviews · doi:10.1039/b510396g · Cited by 194 · cited 1×
“Inorganic–organic hybrid materials 1–3 such as organically templated metal oxides 1 , metal–organic frameworks (MOFs) 2 and organohalide perovskites 4 have been studied for decades, and hydrothermal and (non-aqueous) solvothermal syntheses have produced thousands of new materials that collectively…”
BackgroundEclectic Rule-Extraction From Support Vector Machines
Barakat, N., & Diederich, J. (2008) · doi:10.5281/zenodo.1055511 · Cited by 136
Will it crystallise? Predicting crystallinity of molecular materials
Wicker, J. G. P., & Cooper, R. I. (2014) · CrystEngComm · doi:10.1039/c4ce01912a · Cited by 94 · cited 1×
“A machine-learning approach to the related problem of whether a particular organic molecule will crystallize has been described previously 25 .”
BackgroundMartin, R. L., Lin, L., Jariwala, K., Smit, B., & Harańczyk, M. (2013) · The Journal of Physical Chemistry C · doi:10.1021/jp401920y · Cited by 83 · cited 1×
“…properties 12 , gas adsorption capacity 13 or lithium-ion intercalation 14 ) to identify promising target candidates for synthetic efforts 11,15 ; determination of the structure–property relationship from large bodies of experimental data 16,17 , enabled by integration with high-throughput…”
BackgroundCrystal Structure and Prediction
Thakur, T. S., Dubey, R., & Desiraju, G. R. (2014) · Annual Review of Physical Chemistry · doi:10.1146/annurev-physchem-040214-121452 · Cited by 82 · cited 1×
“First-principles crystal-structure prediction—even for simple crystallization from a solvent—is fundamentally difficult, owing to the need to consider a combinatorially enormous set of component arrangements 22,23 using high-level quantum chemistry methods 24 .”
BackgroundThangavelu, S. G., Butcher, R. J., & Cahill, C. L. (2015) · Crystal Growth & Design · doi:10.1021/acs.cgd.5b00549 · Cited by 54 · cited 1×
“For consistency, human reactions used a rule-based approach that is widely used by the exploratory hydrothermal synthesis community 37 , namely, scaling the masses of the organic amines by their respective formula weights, while all other reaction parameters remain unchanged.”
MethodA New Era for ab initio Molecular Crystal Lattice Energy Prediction
Beran, G. J. O. (2014) · Angewandte Chemie International Edition · doi:10.1002/anie.201409823 · Cited by 43 · cited 1×
“First-principles crystal-structure prediction—even for simple crystallization from a solvent—is fundamentally difficult, owing to the need to consider a combinatorially enormous set of component arrangements 22,23 using high-level quantum chemistry methods 24 .”
BackgroundIdentifying Zeolite Frameworks with a Machine Learning Approach
Yang, S., Lach-hab, M., Vaisman, I. I., & Blaisten‐Barojas, E. (2009) · The Journal of Physical Chemistry C · doi:10.1021/jp907017u · Cited by 39 · cited 1×
“…relationship from large bodies of experimental data 16,17 , enabled by integration with high-throughput synthesis and measurement tools 18 ; and clustering on the basis of similar crystallographic structure (for example, zeolite structure classification 19,20 or gas adsorption properties 21 ).”
BackgroundHigh-throughput experimental tools for the materials genome initiative
Zhao, J. (2014) · Chinese Science Bulletin · doi:10.1007/s11434-014-0120-1 · Cited by 33 · cited 1×
“…relationship from large bodies of experimental data 16,17 , enabled by integration with high-throughput synthesis and measurement tools 18 ; and clustering on the basis of similar crystallographic structure (for example, zeolite structure classification 19,20 or gas adsorption…”
BackgroundSixth blind test of organic crystal-structure prediction methods
Groom, C. R., & Reilly, A. M. (2014) · Acta Crystallographica Section B Structural Science Crystal Engineering and Materials · doi:10.1107/s2052520614015923 · Cited by 15 · cited 1×
“First-principles crystal-structure prediction—even for simple crystallization from a solvent—is fundamentally difficult, owing to the need to consider a combinatorially enormous set of component arrangements 22,23 using high-level quantum chemistry methods 24 .”
BackgroundFormation Principles for Vanadium Selenites: The Role of pH on Product Composition
Olshansky, J. H., Wiener, K. J., Smith, M. D., Nourmahnad, A., Charles, M. J., Zeller, M., Schrier, J., & Norquist, A. J. (2014) · Inorganic Chemistry · doi:10.1021/ic501736r · Cited by 13
Self-citation
Cited by
Machine learning for molecular and materials science
Butler, K. T., Davies, D. W., Cartwright, H., Isayev, O., & Walsh, A. (2018) · Nature · doi:10.1038/s41586-018-0337-2 · Cited by 4,968
“Another study trained a model to predict the reaction conditions for new organically templated inorganic product formation with a success rate of 89%.(45)”
BackgroundData-driven prediction of battery cycle life before capacity degradation
Severson, K. A., Attia, P. M., Jin, N., Perkins, N., Jiang, B., Yang, Z., Chen, M. H., Aykol, M., Herring, P. K., Fraggedakis, D., et al. (2019) · Nature Energy · doi:10.1038/s41560-019-0356-8 · Cited by 3,228
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules
Gómez-Bombarelli, R., Wei, J. N., Duvenaud, D., Hernández-Lobato, J. M., Sánchez-Lengeling, B., Sheberla, D., Aguilera-Iparraguirre, J., Hirzel, T. D., Adams, R. P., & Aspuru-Guzik, A. (2018) · ACS Central Science · doi:10.1021/acscentsci.7b00572 · Cited by 3,106
“Figures 13-15 ) Bayesian optimization of logP Having shown the ability of a VAE to generate a continuous, reversible encoding of molecular structures, we proceeded to test its potential to discover new molecules with desired properties.”
BackgroundRecent advances and applications of machine learning in solid-state materials science
Schmidt, J., Marques, M. R. G., Botti, S., & Marques, M. A. L. (2019) · npj Computational Materials · doi:10.1038/s41524-019-0221-0 · Cited by 2,607
Inverse molecular design using machine learning: Generative models for matter engineering
Sanchez-Lengeling, B., & Aspuru-Guzik, A. (2018) · Science · doi:10.1126/science.aat2663 · Cited by 2,050
Metadata
- Authors
- Raccuglia, P., Elbert, K. C., Adler, P. D. F., Falk, C., Wenny, M. B., Mollo, A., Zeller, M., Friedler, S. A., Schrier, J., & Norquist, A. J.
- Affiliations
- Haverford College (US)
- Corresponding author
- Friedler, Sorelle A.
- Schrier, Joshua
- Norquist, Alexander J.
- ORCID
- Venue
- Nature
- Published
- 2016-05-04
- Volume
- 533
- Issue
- 7601
- Pages
- 73-76
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2347129741
- Cited by
- 1,727
- References
- 37
- Citations by year
- 2026: 215 · 2025: 225 · 2024: 214 · 2023: 169 · 2022: 201 · 2021: 206 · 2020: 190 · 2019: 148 · 2018: 95 · 2017: 51 · 2016: 13
- FWCI
- 45.53
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Drug Discovery › Computational Drug Discovery Methods
- Social Sciences › Arts and Humanities › History and Philosophy of Science › History and advancements in chemistry
- Keywords
- dark reaction, chemical reaction prediction, machine learning, hydrothermal synthesis, laboratory notebooks, cheminformatics, organic-inorganic hybrid materials, materials discovery, crystal structure prediction, synthetic route design, Materials Genome Initiative
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.