CoChat
Sign Up
Cited by 771Topic ModelingMachine Learning in Materials ScienceBiomedical Text Mining and Ontologies

Structured information extraction from scientific text with large language models

Open access
Dagdelen, John, Dunn, Alexander, Lee, Sanghoon, Walker, Nicholas, Rosen, Andrew S., Ceder, Gerbrand, Persson, Kristin A., Jain, Anubhav·2024·Nature Communications
DOI 10.1038/s41467-024-45563-xPMC PMC10869356
Open this paper in CoChat — free accountOpen PDF

Abstract

Extracting structured knowledge from scientific text remains a challenging task for machine learning models. Here, we present a simple approach to joint named entity recognition and relation extraction and demonstrate how pretrained large language models (GPT-3, Llama-2) can be fine-tuned to extract useful records of complex scientific knowledge. We test three representative tasks in materials chemistry: linking dopants and host materials, cataloging metal-organic frameworks, and general composition/phase/morphology/application information extraction. Records are extracted from single sentences or entire paragraphs, and the output can be returned as simple English sentences or a more structured format such as a list of JSON objects. This approach represents a simple, accessible, and highly flexible route to obtaining large databases of structured specialized scientific knowledge extracted from research papers.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-024-45563-x

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Recent systems have shown strong general-purpose language capabilities and encouraging results in scientific text mining, information extraction, and structured generation.

    “Recent systems have shown strong general-purpose language capabilities and encouraging results in scientific text mining, information extraction, and structured generation [7][8][9][10].”

    From Diagnosing the limits of large language models for structured scientific evidence extraction (2026) · cites this paper as [8]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 60

  1. [1]Saal JE Oliynyk AO Meredig B Machine learning in materials discovery: confirmed predictions and their underlying approaches Annu. Rev. Mater. Res. 2020 50 49 69 10.1146/annurev-matsci-090319-010954 · DOI
  2. [3]Oliveira ON Oliveira MCF Materials discovery with machine learning and knowledge discovery Front. Chem. 2022 10 930369 10.3389/fchem.2022.930369 35873055 PMC9300917 · DOI
  3. [4]Weston L Named entity recognition and normalization applied to large-scale information extraction from the materials science literature J. Chem. Inform. Modeling 2019 59 3692 3702 10.1021/acs.jcim.9b00470 31361962 · DOI
  4. [5]Trewartha A Quantifying the advantage of domain-specific pre-training on named entity recognition tasks in materials science Patterns 2022 3 100488 10.1016/j.patter.2022.100488 35465225 PMC9024010 · DOI
  5. [6]Isazawa T Cole JM Single model for organic and inorganic chemical named entity recognition in ChemDataExtractor J. Chem. Inform. Modeling 2022 62 1207 1213 10.1021/acs.jcim.1c01199 PMC9049593 35199519 · DOI
  6. [7]Zhao, X., Greenberg, J., An, Y. & Hu, X. T. Fine-tuning BERT model for materials named entity recognition. In: 2021 IEEE International Conference on Big Data (Big Data) (IEEE, 2021). 10.1109/bigdata52589.2021.9671697. · DOI
  7. [8]Sierepeklis O Cole JM A thermoelectric materials database auto-generated from the scientific literature using chemdataextractor Sci. data 2022 9 648 10.1038/s41597-022-01752-1 36272983 PMC9587980 · DOI
  8. [9]Beard EJ Cole JM Perovskite- and dye-sensitized solar-cell device databases auto-generated using chemdataextractor Sci. Data 2022 9 329 10.1038/s41597-022-01355-w 35715446 PMC9205998 · DOI
  9. [10]Kumar P Kabra S Cole JM Auto-generating databases of yield strength and grain size using chemdataextractor Sci. Data 2022 9 292 10.1038/s41597-022-01301-w · DOI
  10. [11]Huang S Cole JM BatteryBERT: A pretrained language model for battery database enhancement J. Chem. Inform. Modeling 2022 62 6365 6377 10.1021/acs.jcim.2c00035 PMC9795558 35533012 · DOI
  11. [12]Dong Q Cole JM Auto-generated database of semiconductor band gaps using chemdataextractor Sci. Data 2022 9 193 10.1038/s41597-022-01294-6 35504897 PMC9065101 · DOI
  12. [13]Kononova O Text-mined dataset of inorganic materials synthesis recipes Sci. Data 2019 6 203 10.1038/s41597-019-0224-1 31615989 PMC6794279 · DOI
  13. [14]Huo H Machine-learning rationalization and prediction of solid-state synthesis conditions Chem. Mater. 2022 34 7323 7336 10.1021/acs.chemmater.2c01293 36032555 PMC9407029 · DOI
  14. [15]He T Similarity of precursors in solid-state synthesis as text-mined from scientific literature Chem. of Mater. 2020 32 7861 7873 10.1021/acs.chemmater.0c02553 · DOI
  15. [16]Wang Z Dataset of solution-based inorganic materials synthesis procedures extracted from the scientific literature Sci. Data 2022 9 231 10.1038/s41597-022-01317-2 35614129 PMC9132903 · DOI
  16. [17]Huang S Cole JM A database of battery materials auto-generated using ChemDataExtractor Sci. Data 2020 7 260 10.1038/s41597-020-00602-2 32764659 PMC7411033 · DOI
  17. [18]Beard EJ Sivaraman G Vázquez-Mayagoitia Á Vishwanath V Cole JM Comparative dataset of experimental and computational attributes of UV/vis absorption spectra Sci. Data 2019 6 307 10.1038/s41597-019-0306-0 31804487 PMC6895184 · DOI
  18. [19]Zhao J Cole JM A database of refractive indices and dielectric constants auto-generated using ChemDataExtractor Sci. Data 2022 9 192 10.1038/s41597-022-01295-5 35504964 PMC9065060 · DOI
  19. [20]Bekoulis G Deleu J Demeester T Develder C Joint entity recognition and relation extraction as a multi-head selection problem Expert Syst. Appl. 2018 114 34 45 10.1016/j.eswa.2018.07.032 · DOI
  20. [21]Han, X. et al. More data, more relations, more context and more openness: a review and outlook for relation extraction. In: Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, 745–758. (Association for Computational Linguistics, 2020). https://aclanthology.org/2020.aacl-main.75.
  21. [22]Yao, Y. et al. DocRED: A large-scale document-level relation extraction dataset. In: Proc. 57th Annual Meeting of the Association for Computational Linguistics, 764–777 (Association for Computational Linguistics, 2019). https://aclanthology.org/P19-1074.
  22. [23]Li, J. et al. Biocreative v cdr task corpus: a resource for chemical disease relation extraction. Database2016 (2016). PMC4860626 27161011
  23. [24]Bravo Á Piñero J Queralt-Rosinach N Rautschka M Furlong LI Extraction of relations between genes and diseases from text and large-scale data analysis: Implications for translational research BMC Bioinformatics 2015 16 1 17 10.1186/s12859-015-0472-9 25886734 PMC4466840 · DOI
  24. [25]Yang, X. et al. Pcmsp: A dataset for scientific action graphs extraction from polycrystalline materials synthesis procedure text. In: Findings of the Association for Computational Linguistics: EMNLP 2022, 6033–6046 (Association for Computational Linguistics, 2022). https://aclanthology.org/2022.findings-emnlp.446.
  25. [26]Brown, T. B. et al. Language models are few-shot learners. Preprint at https://browse.arxiv.org/abs/2005.14165 (2020). · arXiv
  26. [27]OpenAI. Gpt-4 technical report. Preprint at https://browse.arxiv.org/abs/2303.08774 (2023). · arXiv
  27. [30]Touvron, H. et al. Llama: Open and efficient foundation language models. Preprint at https://arxiv.org/abs/2302.13971 (2023). · arXiv
  28. [31]Touvron, H. et al. Llama 2: Open foundation and fine-tuned chat models. Preprint at https://arxiv.org/abs/2307.09288 (2023). · arXiv
  29. [32]Zhang, S. et al. Opt: Open pre-trained transformer language models. Preprint at https://browse.arxiv.org/abs/2205.01068 (2022). · arXiv
  30. [33]Hoffmann, J. et al. Training compute-optimal large language models. Preprint at http://arxiv.org/abs/2203.15556 (2022). · arXiv
  31. [34]Wei, J. et al. Finetuned language models are zero-shot learners. In: International Conference on Learning Representations (2022). https://openreview.net/forum?id=gEZrGCozdqR.
  32. [36]Dabre R Chu C Kunchukuttan A A survey of multilingual neural machine translation ACM Comput. Surv. 2020 53 1 38 10.1145/3406095 · DOI
  33. [37]Petroni, F. et al. Language models as knowledge bases? In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2463–2473 (Association for Computational Linguistics, 2019). https://aclanthology.org/D19-1250.
  34. [38]Han, J. M. et al. Unsupervised neural machine translation with generative language models only. https://openreview.net/forum?id=SVwbKmEg7M (2022).
  35. [39]Zhang, H., Xu, J. & Wang, J. Pretraining-based natural language generation for text summarization. In: Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), 789–797 (Association for Computational Linguistics, 2019). https://aclanthology.org/K19-1074.
  36. [40]Liu, Z. et al. Multi-stage prompting for knowledgeable dialogue generation. In: Findings of the Association for Computational Linguistics: ACL 2022, 1317–1337 (Association for Computational Linguistics, 2022). https://aclanthology.org/2022.findings-acl.104.
  37. [41]Giorgi, J., Bader, G. & Wang, B. A sequence-to-sequence approach for document-level relation extraction. In: Proc. 21st Workshop on Biomedical Language Processing, 10–25 (Association for Computational Linguistics, 2022). https://aclanthology.org/2022.bionlp-1.2.
  38. [42]Cabot, P.-L. H. & Navigli, R. REBEL: Relation extraction by end-to-end language generation. In: Findings of the Association for Computational Linguistics: EMNLP 2021 (Association for Computational Linguistics, 2021). 10.18653/v1/2021.findings-emnlp.204. · DOI · Paper page
  39. [43]Townsend, B., Ito-Fisher, E., Zhang, L. & May, M. Doc2dict: Information extraction as text generation. Preprint at http://arxiv.org/abs/2105.07510 (2021). · arXiv
  40. [44]Zheng Z Zhang O Borgs C Chayes JT Yaghi OM ChatGPT chemistry assistant for text mining and the prediction of MOF synthesis J. Am. Chem. Soc. 2023 145 18048 18062 10.1021/jacs.3c05819 37548379 PMC11073615 · DOI
  41. [46]Castro Nascimento CM Pimentel AS Do large language models understand chemistry? a conversation with chatgpt J. Chem. Inform. Modeling 2023 63 1649 1655 10.1021/acs.jcim.3c00285 36926868 · DOI · Paper page
  42. [47]White AD Assessment of chemistry knowledge in large language models that generate code Digital Discov. 2023 2 368 376 10.1039/D2DD00087C PMC10087057 37065678 · DOI
  43. [48]Xie, T. et al. Darwin series: Domain specific large language models for natural science. Preprint at https://arxiv.org/abs/2308.13565 (2023). · arXiv
  44. [49]Jablonka, K. M., Schwaller, P., Ortega-Guerrero, A. & Smit, B. Is GPT all you need for low-data discovery in chemistry? 10.26434/chemrxiv-2023-fw8n4-v2 (2023). · DOI
  45. [50]Jablonka, K. M. et al. 14 examples of how llms can transform materials science and chemistry: a reflection on a large language model hackathon. Digital Discov. 2, 1233–1250 (2023). PMC10561547 38013906
  46. [51]Gu, Y. et al. Domain-specific language model pretraining for biomedical natural language processing. ACM Trans. Comput. Healthcare3, 1 (2021).
  47. [53]Maynez, J., Narayan, S., Bohnet, B. & McDonald, R. On faithfulness and factuality in abstractive summarization. In: Proc. 58th Annual Meeting of the Association for Computational Linguistics (Association for Computational Linguistics, 2020). 10.18653/v1/2020.acl-main.173. · DOI · Paper page
  48. [55]Black, S. et al. Gpt-neox-20b: An open-source autoregressive language model. Preprint at https://browse.arxiv.org/abs/2204.06745 (2022). · arXiv
  49. [56]Frantar, E. & Alistarh, D. Sparsegpt: Massive language models can be accurately pruned in one-shot. In: Proceedings of the 40th International Conference on Machine Learning, 10323–10337 (JLMR.org, 2023). https://proceedings.mlr.press/v202/frantar23a/frantar23a.pdf.
  50. [57]Sun, M., Liu, Z., Bair, A. & Kolter, J. Z. A simple and effective pruning approach for large language models. In: Workshop on Efficient Systems for Foundation Models @ ICML2023. https://openreview.net/forum?id=tz9JV2PRSv (2023).
  51. [58]Hu, E. J. et al. Lora: Low-rank adaptation of large language models. Preprint at https://arxiv.org/abs/2106.09685 (2021). · arXiv
  52. [59]Ma, X., Fang, G. & Wang, X. Llm-pruner: On the structural pruning of large language models. In: Thirty-seventh Conference on Neural Information Processing Systems (2023). https://openreview.net/forum?id=J8Ajf9WfXP.
  53. [60]Bucior BJ Identification schemes for metal–organic frameworks to enable rapid search and cheminformatics analysis Cryst. Growth Des. 2019 19 6682 6697 10.1021/acs.cgd.9b01050 · DOI
  54. [61]Li X Hill MR Wang H Zhang H Metal–organic framework-based ion-selective membranes Adv. Mater. Technol. 2021 6 2000790 10.1002/admt.202000790 · DOI
  55. [62]Qian Q Mof-based membranes for gas separations Chem. Rev. 2020 120 8161 8266 10.1021/acs.chemrev.0c00119 32608973 · DOI
  56. [63]Tshitoyan V Unsupervised word embeddings capture latent knowledge from materials science literature Nature 2019 571 95 98 10.1038/s41586-019-1335-8 31270483 · DOI · Paper page
  57. [64]Kluyver, T. et al. Jupyter notebooks—a publishing format for reproducible computational workflows. In: Positioning and Power in Academic Publishing: Players, Agents and Agendas (eds. Loizides, F. & Schmidt, B.) 87–90 (IOS Press, 2016).
  58. [65]Dagdelen, J. et al. Structured information extraction from scientific text with large language models 10.5281/zenodo.10421174 (2023). PMC10869356 38360817 · DOI
  59. [66]Dagdelen, J. et al. Llama 2 code for structured information extraction from scientific text with large language models. 10.5281/zenodo.10421187 (2023). PMC10869356 38360817 · DOI
  60. [67]Dagdelen, J. et al. Llama 2 weights for structured information extraction from scientific text with large language models. 10.6084/m9.figshare.24501331.v1 (2023). PMC10869356 38360817 · DOI

Cited by

Sorted by citations

Authors
Dagdelen, J., Dunn, A., Lee, S., Walker, N., Rosen, A. S., Ceder, G., Persson, K. A., & Jain, A.
Affiliations
Lawrence Berkeley National Laboratory (US)
Corresponding author
Jain, Anubhav
ORCID
Venue
Nature Communications
Publisher
Springer Science and Business Media LLC
Published
2024-02-15
Volume
15
Issue
1
Pages
1418-1418
ISSN
2041-1723 (ISSN-L)
DOI
10.1038/s41467-024-45563-x
PMC
PMC10869356
OpenAlex
W4391836235
Cited by
771
References
4
Citations by year
2026: 368 · 2025: 308 · 2024: 91 · 2023: 4
FWCI
112.47
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • eScholarship (California Digital Library) (submittedVersion, cc-by)
  • OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion, other-oa)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • U.S. Department of Energy (-AC02-05CH11231, BES-ERCAP0024004, 05CH11231, DE AC02 05CH11231, DE-AC02-)
  • Adolph C. and Mary Sprague Miller Institute for Basic Research in Science, University of California Berkeley
  • Toyota Research Institute
  • National Energy Research Scientific Computing Center (DE–AC02–05CH11231, 05CH11231, AC02-05CH11231, BES-ERCAP0024004)
  • Office of Science (DEAC02–05CH11231, AC02-05CH11231, DE-AC02, 05CH11231)
  • Basic Energy Sciences (DE-AC02, 05CH11231, DE-AC02-05CH11231., KCD2S2, AC02–05CH11231)
  • Lawrence Berkeley National Laboratory (AC02‐05CH11231, DE-AC02-05CH11231, 05CH11231)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
Keywords
structured data extraction, scientific texts, named entity recognition, relation extraction, large language models, GPT-3, Llama 2, fine-tuning, materials chemistry, metal-organic frameworks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review