ChemDataExtractor: A Toolkit for Automated Extraction of Chemical Information from the Scientific Literature
Open accessAbstract
The emergence of "big data" initiatives has led to the need for tools that can automatically extract valuable chemical information from large volumes of unstructured data, such as the scientific literature. Since chemical information can be present in figures, tables, and textual paragraphs, successful information extraction often depends on the ability to interpret all of these domains simultaneously. We present a complete toolkit for the automated extraction of chemical entities and their associated properties, measurements, and relationships from scientific documents that can be used to populate structured chemical databases. Our system provides an extensible, chemistry-aware, natural language processing pipeline for tokenization, part-of-speech tagging, named entity recognition, and phrase parsing. Within this scope, we report improved performance for chemical named entity recognition through the use of unsupervised word clustering based on a massive corpus of chemistry articles. For phrase parsing and information extraction, we present the novel use of multiple rule-based grammars that are tailored for interpreting specific document domains such as textual paragraphs, captions, and tables. We also describe document-level processing to resolve data interdependencies and show that this is particularly necessary for the autogeneration of chemical databases since captions and tables commonly contain chemical identifiers and references that are defined elsewhere in the text. The performance of the toolkit to correctly extract various types of data was evaluated, affording an F-score of 93.4%, 86.8%, and 91.5% for extracting chemical identifiers, spectroscopic attributes, and chemical property attributes, respectively; set against the CHEMDNER chemical name extraction challenge, ChemDataExtractor yields a competitive F-score of 87.8%. All tools have been released under the MIT license and are available to download from http://www.chemdataextractor.org .
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In experimental data sets extracted from the literature, there can be positive bias for reporting only successful experiments, leading to challenges in classification tasks.
“For example, in experimental data sets extracted from the literature (14,15,19), there can be positive bias for reporting only successful experiments (16, 107, 108), leading to challenges in classification tasks (17).”
From Machine Learning for the Discovery, Design, and Engineering of Materials. (2022) · cites this paper as [19]
Cites
Commentary: The Materials Project: A materials genome approach to accelerating materials innovation
Jain, A., Ong, S. P., Hautier, G., Chen, W., Richards, W. D., Dacek, S., Cholia, S., Gunter, D., Skinner, D., Ceder, G., et al. (2013) · APL Materials · doi:10.1063/1.4812323 · Cited by 13,094 · see citing works
(1992) · doi:10.5555/176313.176316 · Cited by 2,892 · see citing works
(2010) · Cited by 1,906 · see citing works
Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters
Owoputi, O., O'Connor, B., Dyer, C., Gimpel, K., Schneider, N., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473408 · Cited by 662 · see citing works
ANTLR: A predicated‐ LL ( k ) parser generator
Parr, T. J., & Quong, R. W. (1995) · Software Practice and Experience · doi:10.1002/spe.4380250705 · Cited by 619 · see citing works
RelEx—Relation extraction using dependency parse trees
Fundel, K., Küffner, R., & Zimmer, R. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl616 · Cited by 569 · see citing works
A SIMPLE ALGORITHM FOR IDENTIFYING ABBREVIATION DEFINITIONS IN BIOMEDICAL TEXT
SCHWARTZ, A. S., & Hearst, M. A. (2002) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789812776303_0042 · Cited by 509 · see citing works
Developing a Robust Part-of-Speech Tagger for Biomedical Text
Tsuruoka, Y., Tateishi, Y., Kim, J.-D., Ohta, T., McNaught, J., Ananiadou, S., & Tsujii, J. (2005) · Lecture notes in computer science · doi:10.1007/11573036_36 · Cited by 439 · see citing works
Unsupervised Multilingual Sentence Boundary Detection
Kiss, T., & Strunk, J. (2006) · Computational Linguistics · doi:10.1162/coli.2006.32.4.485 · Cited by 423 · see citing works
Materials Genome Initiative for Global Competitiveness
Ward, C. (2012) · Cited by 408 · see citing works
The CHEMDNER corpus of chemicals and drugs and its annotation principles
Krallinger, M., Rabal, O., Leitner, F., Vázquez, M., Salgado, D., Lu, Z., Leaman, R., Lu, Y., Ji, D., Lowe, D. M., et al. (2015) · Journal of Cheminformatics · doi:10.1186/1758-2946-7-s1-s2 · Cited by 406 · see citing works
Frontiers of biomedical text mining: current progress
Zweigenbaum, P., Demner‐Fushman, D., Yu, H. Q., & Cohen, K. B. (2007) · Briefings in Bioinformatics · doi:10.1093/bib/bbm045 · Cited by 282 · see citing works
tmChem: a high performance approach for chemical named entity recognition and normalization
Leaman, R., Wei, C.-H., & Lu, Z. (2015) · Journal of Cheminformatics · doi:10.1186/1758-2946-7-s1-s3 · Cited by 277 · see citing works
ChemSpot: a hybrid system for chemical named entity recognition
Rocktäschel, T., Weidlich, M., & Leser, U. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts183 · Cited by 269 · see citing works
CHEMDNER: The drugs and chemical names extraction challenge
Krallinger, M., Leitner, F., Rabal, O., Vázquez, M., Oyarzábal, J., & Valencia, A. (2015) · Journal of Cheminformatics · doi:10.1186/1758-2946-7-s1-s1 · Cited by 260 · see citing works
OSCAR4: a flexible architecture for chemical text-mining
Jessop, D. M., Adams, S., Willighagen, E., Hawizy, L., & Murray‐Rust, P. (2011) · Journal of Cheminformatics · doi:10.1186/1758-2946-3-41 · Cited by 214 · see citing works
Olivares‐Amaya, R., Amador‐Bedolla, C., Hachmann, J., Atahan-Evrenk, Ş., Sánchez‐Carrera, R. S., Vogt-Maranto, L., & Aspuru‐Guzik, A. (2011) · Energy & Environmental Science · doi:10.1039/c1ee02056k · Cited by 204 · see citing works
ChemicalTagger: A tool for semantic text-mining in chemistry
Hawizy, L., Jessop, D. M., Adams, N., & Murray‐Rust, P. (2011) · Journal of Cheminformatics · doi:10.1186/1758-2946-3-17 · Cited by 192 · see citing works
Filippov, I. V., & Nicklaus, M. C. (2009) · Journal of Chemical Information and Modeling · doi:10.1021/ci800067r · Cited by 189 · see citing works
A dictionary to identify small molecules and drugs in free text
Hettne, K. M., Stierum, R. H., Schuemie, M. J., Hendriksen, P. J. M., Schijvenaars, B. J. A., van Mulligen, E. M., Kleinjans, J. C. S., & Kors, J. A. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp535 · Cited by 162 · see citing works
Chemical named entities recognition: a review on approaches and applications
Eltahier, S. E. M., & Salim, N. B. (2014) · Journal of Cheminformatics · doi:10.1186/1758-2946-6-17 · Cited by 125 · see citing works
Biomedical Text Mining: A Survey of Recent Progress
Simpson, M. S., & Demner‐Fushman, D. (2012) · Springer eBooks · doi:10.1007/978-1-4614-3223-4_14 · Cited by 96 · see citing works
Tetko, I. V., Lowe, D. M., & Williams, A. J. (2016) · Journal of Cheminformatics · doi:10.1186/s13321-016-0113-y · Cited by 89 · see citing works
Text Mining for Drugs and Chemical Compounds: Methods, Tools and Applications
Vázquez, M., Krallinger, M., Leitner, F., & Valencia, A. (2011) · Molecular Informatics · doi:10.1002/minf.201100005 · Cited by 87 · see citing works
LeadMine: a grammar and dictionary driven approach to entity recognition
Lowe, D. M., & Sayle, R. A. (2015) · Journal of Cheminformatics · doi:10.1186/1758-2946-7-s1-s5 · Cited by 66 · see citing works
Sentence Boundary Detection: A Long Solved Problem?
Read, J., Dridan, R., Oepen, S., & Solberg, L. J. (2012) · Cited by 55 · see citing works
Part-of-Speech Annotation of Biology Research Abstracts
Tateisi, Y., & Tsujii, J. (2004) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4kpxeameezza · Cited by 48 · see citing works
CHEMDNER system with mixed conditional random fields and multi-scale word clustering
Lu, Y., Ji, D., Xiaoyuan, Y., Wei, X., & Liang, X. (2015) · Journal of Cheminformatics · doi:10.1186/1758-2946-7-s1-s4 · Cited by 44 · see citing works
Challenges in mining the literature for chemical information
Gurulingappa, H., Mudi, A., Toldo, L., Hofmann‐Apitius, M., & Bhate, J. (2013) · RSC Advances · doi:10.1039/c3ra40787j · Cited by 30 · see citing works
ChemEx: information extraction system for chemical data curation
Tharatipyakul, A., Numnark, S., Wichadakul, D., & Ingsriswang, S. (2012) · BMC Bioinformatics · doi:10.1186/1471-2105-13-s17-s9 · Cited by 28 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Recent advances and applications of deep learning methods in materials science
Choudhary, K., DeCost, B., Chen, C., Jain, A., Tavazza, F., Cohn, R., Park, C. W., Choudhary, A., Agrawal, A., Billinge, S. J. L., et al. (2022) · npj Computational Materials · doi:10.1038/s41524-022-00734-6 · Cited by 1,172 · see citing works
“org [303] ChemicalTagger https://github.”
MethodUnsupervised word embeddings capture latent knowledge from materials science literature
Tshitoyan, V., Dagdelen, J., Weston, L., Dunn, A., Rong, Z., Kononova, O., Persson, K. A., Ceder, G., & Jain, A. (2019) · Nature · doi:10.1038/s41586-019-1335-8 · Cited by 1,123 · see citing works
Data‐Driven Materials Science: Status, Challenges, and Perspectives
Himanen, L., Geurts, A., Foster, A. S., & Rinke, P. (2019) · Advanced Science · doi:10.1002/advs.201900808 · Cited by 808 · see citing works
A Critical Review of Machine Learning of Energy Materials
Chen, C., Zuo, Y., Ye, W., Li, X., Deng, Z., & Ong, S. P. (2020) · Advanced Energy Materials · doi:10.1002/aenm.201903242 · Cited by 609 · see citing works
Opportunities and Challenges for Machine Learning in Materials Science
Morgan, D., & Jacobs, R. (2020) · Annual Review of Materials Research · doi:10.1146/annurev-matsci-070218-010015 · Cited by 549 · see citing works
Materials Synthesis Insights from Scientific Literature via Text Extraction and Machine Learning
Kim, E., Huang, K., Saunders, A., McCallum, A., Ceder, G., & Olivetti, E. (2017) · Chemistry of Materials · doi:10.1021/acs.chemmater.7b03500 · Cited by 473 · see citing works
Pyzer-Knapp, E. O., Pitera, J. W., Staar, P. W. J., Takeda, S., Laino, T., Sanders, D. P., Sexton, J., Smith, J. R., & Curioni, A. (2022) · npj Computational Materials · doi:10.1038/s41524-022-00765-z · Cited by 465 · see citing works
Emerging materials intelligence ecosystems propelled by machine learning
Batra, R., Song, L., & Ramprasad, R. (2020) · Nature Reviews Materials · doi:10.1038/s41578-020-00255-y · Cited by 446 · see citing works
Polak, M. P., & Morgan, D. (2024) · Nature Communications · doi:10.1038/s41467-024-45914-8 · Cited by 365 · see citing works
Nandy, A., Duan, C., Taylor, M. G., Liu, F., Steeves, A. H., & Kulik, H. J. (2021) · Chemical Reviews · doi:10.1021/acs.chemrev.1c00347 · Cited by 358 · see citing works
Data-driven materials research enabled by natural language processing and information extraction
Olivetti, E. A., Cole, J. M., Kim, E., Kononova, O., Ceder, G., Han, T. Y.-J., & Hiszpanski, A. M. (2020) · Applied Physics Reviews · doi:10.1063/5.0021106 · Cited by 302 · see citing works
Self-citationInformation retrieval and text mining technologies for chemistry
Krallinger, M., Rabal, O., Lourenço, A., Oyarzabal, J., & Valencia, A. (2017) · Chemical Reviews · doi:10.1021/acs.chemrev.6b00851 · Cited by 281 · see citing works
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274 · see citing works
Accelerating materials discovery using machine learning
Juan, Y., Dai, Y., Yang, Y., & Zhang, J. (2021) · Journal of Materials Science & Technology · doi:10.1016/j.jmst.2020.12.010 · Cited by 240 · see citing works
Nandy, A., Duan, C., & Kulik, H. J. (2021) · Journal of the American Chemical Society · doi:10.1021/jacs.1c07217 · Cited by 200 · see citing works
Opportunities and challenges of text mining in materials research
Kononova, O., He, T., Huo, H., Trewartha, A., Olivetti, E. A., & Ceder, G. (2021) · iScience · doi:10.1016/j.isci.2021.102155 · Cited by 182 · see citing works
Semi-supervised machine-learning classification of materials synthesis procedures
Huo, H., Rong, Z., Kononova, O., Sun, W., Botari, T., He, T., Tshitoyan, V., & Ceder, G. (2019) · npj Computational Materials · doi:10.1038/s41524-019-0204-1 · Cited by 151 · see citing works
Applications of natural language processing and large language models in materials discovery
Jiang, X., Wang, W., Tian, S., Wang, H., Lookman, T., & Su, Y. (2025) · npj Computational Materials · doi:10.1038/s41524-025-01554-0 · Cited by 145 · see citing works
Machine Learning for the Discovery, Design, and Engineering of Materials.
Duan, C., Nandy, A., & Kulik, H. J. (2022) · Annual Review of Chemical and Biomolecular Engineering · doi:10.1146/annurev-chembioeng-092320-120230 · Cited by 69 · see citing works
Machine learning in materials design: Algorithm and application*
Song, Z., Chen, X., Meng, F., Cheng, G., Wang, C., Sun, Z., & Yin, W.-J. (2020) · Chinese Physics B · doi:10.1088/1674-1056/abc0e3 · Cited by 63 · see citing works
Ravinder, Venugopal, V., Bishnoi, S., Singh, S., Zaki, M., Grover, H. S., Bauchy, M., Agarwal, M., & Krishnan, N. M. A. (2021) · International Journal of Applied Glass Science · doi:10.1111/ijag.15881 · Cited by 56 · see citing works
Decoding the Foodome: Molecular Networks Connecting Diet and Health
Menichetti, G., Barabási, A.-L., & Loscalzo, J. (2024) · Annual Review of Nutrition · doi:10.1146/annurev-nutr-062322-030557 · Cited by 32 · see citing works
Language models for materials discovery and sustainability: Progress, challenges, and opportunities
Pei, Z., Yin, J., & Zhang, J. (2025) · Progress in Materials Science · doi:10.1016/j.pmatsci.2025.101495 · Cited by 30 · see citing works
- Authors
- Swain, M. C., & Cole, J. M.
- Affiliations
- University of Cambridge (GB)
- Corresponding author
- Cole, Jacqueline M.
- Venue
- Journal of Chemical Information and Modeling
- Publisher
- American Chemical Society (ACS)
- Published
- 2016-10-06
- Volume
- 56
- Issue
- 10
- Pages
- 1894-1904
- ISSN
- 1549-9596 (ISSN-L)
- 1549-960X
- OpenAlex
- W2523785361
- Cited by
- 598
- References
- 30
- Citations by year
- 2026: 93 · 2025: 103 · 2024: 80 · 2023: 99 · 2022: 65 · 2021: 63 · 2020: 40 · 2019: 34 · 2018: 13 · 2017: 8
- FWCI
- 9.53
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Locations
- Journal of Chemical Information and Modeling (publishedVersion)
- PubMed (publishedVersion)
- Apollo (University of Cambridge) (submittedVersion)
- Apollo
- Funders
- Engineering and Physical Sciences Research Council (EP/J500380/1)
- Royal Commission for the Exhibition of 1851
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- chemical information extraction, chemical named entity recognition, scientific publications, rule-based grammar, chemical identifiers
- MeSH terms
- Humans, Natural Language Processing, Pattern Recognition, Automated, Periodicals as Topic, Cluster Analysis, Data Mining, Databases, Chemical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
