Cited by 82Machine Learning in Materials ScienceTopic ModelingMultimodal Machine Learning Applications

Generative Retrieval-Augmented Ontologic Graph and Multiagent Strategies for Interpretive Large Language Model-Based Materials Design

Open access
Buehler, Markus J.·2024·ACS Engineering Au

Abstract

Transformer neural networks show promising capabilities, in particular for uses in materials analysis, design, and manufacturing, including their capacity to work effectively with human language, symbols, code, and numerical data. Here, we explore the use of large language models (LLMs) as a tool that can support engineering analysis of materials, applied to retrieving key information about subject areas, developing research hypotheses, discovery of mechanistic relationships across disparate areas of knowledge, and writing and executing simulation codes for active knowledge generation based on physical ground truths. Moreover, when used as sets of AI agents with specific features, capabilities, and instructions, LLMs can provide powerful problem-solution strategies for applications in analysis and design problems. Our experiments focus on using a fine-tuned model, MechGPT, developed based on training data in the mechanics of materials domain. We first affirm how fine-tuning endows LLMs with a reasonable understanding of subject area knowledge. However, when queried outside the context of learned matter, LLMs can have difficulty recalling correct information and may hallucinate. We show how this can be addressed using retrieval-augmented Ontological Knowledge Graph strategies. The graph-based strategy helps us not only to discern how the model understands what concepts are important but also how they are related, which significantly improves generative performance and also naturally allows for injection of new and augmented data sources into generative AI algorithms. We find that the additional feature of relatedness provides advantages over regular retrieval augmentation approaches and not only improves LLM performance but also provides mechanistic insights for exploration of a material design process. Illustrated for a use case of relating distinct areas of knowledge, here, music and proteins, such strategies can also provide an interpretable graph structure with rich information at the node, edge, and subgraph level that provides specific insights into mechanisms and relationships. We discuss other approaches to improve generative qualities, including nonlinear sampling strategies and agent-based modeling that offer enhancements over single-shot generations, whereby LLMs are used to both generate content and assess content against an objective target. Examples provided include complex question answering, code generation, and execution in the context of automated force-field development from actively learned density functional theory (DFT) modeling and data analysis.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1021/acsengineeringau.3c00058

Cites

103 of 103 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The remaining literature was distributed across general materials-workflow papers, atomistic simulation infrastructures, MOF and porous-material screening, and polymer simulation.

    “The remaining literature was distributed across general materials-workflow papers, atomistic simulation infrastructures, MOF and porous-material screening, and polymer simulation [17,21,22,24-26,32,35-42].”
  • Other papers were more demonstrative, emphasizing workflow feasibility, design navigation, or case-based success rather than tightly standardized metrics.

    “Other papers were more demonstrative, emphasizing workflow feasibility, design navigation, or case-based success rather than tightly standardized metrics [17,18,21,22,35,37-39,41].”
  • In other papers, the workflow could be reconstructed only partially from supporting information, and prompt logs, failure traces, or decision histories remained incomplete.

    “In others, the workflow could be reconstructed only partially from supporting information, and prompt logs, failure traces, or decision histories remained incomplete [17,22,24,26,27,29,33,34].”

    From From Scientific Copilots to Tool-Grounded Autonomy: AI Agents in Simulation-Driven Materials Discovery (2026) · cites this paper as [17]

    The Results and Discussion section describes agent workflows and says Table 9 summarizes a conversation; the Figure 8 caption says generated code is stored for later retrieval. Searches for supporting information, prompt logs, failure traces, and decision histories found no passage assessing whether those records were incomplete or the workflow could only be partially reconstructed. The cited paper’s text therefore does not establish this claim about its reporting. (LLM, conf=0.70)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. Ge Y.; Hua W.; Mei K.; Ji J.; Tan J.; Xu S.; Li Z.; Zhang Y. OpenAGI: When LLM Meets Domain Experts, (2023). arXiv:2304.04370v5. arXiv.org e-Print archive. https://arxiv.org/abs/2304.04370v5 (accessed September 23, 2023). · arXiv
  2. OpenAI, GPT-4 Technical Report, (2023). arxiv:2303.08774v3. arXiv.org e-Print archive. https://arxiv.org/abs/2303.08774v3 (accessed September 23, 2023). · arXiv
  3. Cranford S. W.; Buehler M. J. Biomateriomics; Springer: Netherlands, 2012.
  4. van der Zant T.; Kouw M.; Schomaker L. Generative artificial intelligence. In Studies in Applied Philosophy, Epistemology and Rational Ethics; Springer: Berlin Heidelberg, 2013; Vol. 5, pp 107– 120. 10.1007/978-3-642-31674-6_8/COVER. · DOI
  5. Anderson P. M.; Hirth J. P.; Lothe J. Theory of Dislocations, Third ed.; Cambridge University Press, 2017; p 1543.

Cited by

Sorted by citations

Authors
Buehler, M. J.
Affiliations
Massachusetts Institute of Technology (US)
Corresponding author
Buehler, Markus J.
Venue
ACS Engineering Au
Published
2024-01-12
Volume
4
Issue
2
Pages
241-277
ISSN
2694-2488 (ISSN-L)
OpenAlex
W4390824394
Cited by
82
References
73
Citations by year
2026: 36 · 2025: 28 · 2024: 18
FWCI
8.00
Type
journal article
Language
en
Open access
Locations
  • ACS Engineering Au (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
  • Army Research Office (W911NF1920098, W911NF2220213)
  • Massachusetts Institute of Technology
  • Office of Naval Research (N00014-19-1-2375, N00014-20-1-2189)
  • National Institutes of Health
  • MIT-IBM Watson AI Lab
  • U.S. Department of Agriculture (2021-69012-35978)
Topics
  • Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
large language models, materials design, retrieval-augmented generation, multi-agent systems, fine-tuning, hallucination mitigation, code generation, program execution, force field development, density functional theory, nonlinear sampling, question answering

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review