The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Anthropogenic biases in chemical reaction data hinder exploratory inorganic synthesis
Abstract
Most chemical experiments are planned by human scientists and therefore are subject to a variety of human cognitive biases 1, heuristics 2 and social influences 3. These anthropogenic chemical reaction data are widely used to train machine-learning models 4 that are used to predict organic 5 and inorganic 6,7 syntheses. However, it is known that societal biases are encoded in datasets and are perpetuated in machine-learning models 8. Here we identify as-yet-unacknowledged anthropogenic biases in both the reagent choices and reaction conditions of chemical reaction datasets using a combination of data mining and experiments. We find that the amine choices in the reported crystal structures of hydrothermal synthesis of amine-templated metal oxides 9 follow a power-law distribution in which 17% of amine reactants occur in 79% of reported compounds, consistent with distributions in social influence models 10-12. An analysis of unpublished historical laboratory notebook records shows similarly biased distributions of reaction condition choices. By performing 548 randomly generated experiments, we demonstrate that the popularity of reactants or the choices of reaction conditions are uncorrelated to the success of the reaction. We show that randomly generated experiments better illustrate the range of parameter choices that are compatible with crystal formation. Machine-learning models that we train on a smaller randomized reaction dataset outperform models trained on larger human-selected reaction datasets, demonstrating the importance of identifying and addressing anthropogenic biases in scientific data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, in experimental data sets extracted from the literature (14,15,19), there can be positive bias for reporting only successful experiments (16, 107, 108), leading to challenges in classification tasks (17).”
From Machine Learning for the Discovery, Design, and Engineering of Materials. (2022) · cites this paper as [107]
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
The Cambridge Structural Database
Groom, C. R., Bruno, I. J., Lightfoot, M. P., & Ward, S. C. (2016) · Acta Crystallographica Section B Structural Science, Crystal Engineering and Materials · doi:10.1107/s2052520616003954 · Cited by 10,297
Gigerenzer, G., & Gaissmaier, W. (2011) · Annual Review of Psychology · doi:10.1146/annurev-psych-120709-145346 · Cited by 4,037
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Conditions for intuitive expertise: A failure to disagree.
Kahneman, D., & Klein, G. (2009) · American Psychologist · doi:10.1037/a0016755 · Cited by 2,443
Open-Framework Inorganic Materials
Cheetham, A. K., Férey, G., & Loiseau, T. (1999) · Angewandte Chemie International Edition · doi:10.1002/(sici)1521-3773(19991115)38:22 · Cited by 2,431
Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market
Salganik, M. J., Dodds, P. S., & Watts, D. J. (2006) · Science · doi:10.1126/science.1121066 · Cited by 2,231
Cited by
Sorted by citations
Big-Data Science in Porous Materials: Materials Genomics and Machine Learning
Jablonka, K. M., Ongari, D., Moosavi, S. M., & Smit, B. (2020) · Chemical Reviews · doi:10.1021/acs.chemrev.0c00004 · Cited by 656
Understanding the diversity of the metal-organic framework ecosystem
Moosavi, S. M., Nandy, A., Jablonka, K. M., Ongari, D., Janet, J. P., Boyd, P. G., Lee, Y., Smit, B., & Kulik, H. J. (2020) · Nature Communications · doi:10.1038/s41467-020-17755-8 · Cited by 622
A Critical Review of Machine Learning of Energy Materials
Chen, C., Zuo, Y., Ye, W., Li, X., Deng, Z., & Ong, S. P. (2020) · Advanced Energy Materials · doi:10.1002/aenm.201903242 · Cited by 609
Machine Learning for Electrocatalyst and Photocatalyst Design and Discovery
Mai, H., Le, T. C., Chen, D., Winkler, D. A., & Caruso, R. A. (2022) · Chemical Reviews · doi:10.1021/acs.chemrev.2c00061 · Cited by 588
Best practices in machine learning for chemistry
Artrith, N., Butler, K. T., Coudert, F., Han, S., Isayev, O., Jain, A., & Walsh, A. (2021) · Nature Chemistry · doi:10.1038/s41557-021-00716-z · Cited by 546
Metadata
- Authors
- Jia, X., Lynch, A., Huang, Y., Danielson, M., Lang’at, I., Milder, A., Ruby, A. E., Wang, H., Friedler, S. A., Norquist, A. J., & Schrier, J.
- Affiliations
- Haverford College (US)
- Corresponding author
- Friedler, Sorelle A.
- Norquist, Alexander J.
- Schrier, Joshua
- ORCID
- Venue
- Nature
- Published
- 2019-09-11
- Volume
- 573
- Issue
- 7773
- Pages
- 251-255
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2972597827
- Cited by
- 222
- References
- 38
- Citations by year
- 2026: 30 · 2025: 18 · 2024: 30 · 2023: 32 · 2022: 32 · 2021: 39 · 2020: 34 · 2019: 7
- FWCI
- 10.11
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Arts and Humanities › History and Philosophy of Science › History and advancements in chemistry
- Keywords
- inorganic synthesis, hydrothermal synthesis, social influence, laboratory notebooks, dataset representativeness
- MeSH terms
- Machine Learning, Humans, Bias, Laboratory Personnel, Chemistry Techniques, Synthetic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
