The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Traceable LLM-based validation of statements in knowledge graphs
Abstract
This article presents a method for verifying RDF triples using LLMs , with an emphasis on providing traceable arguments. Because the LLMs cannot currently reliably identify the origin of the information used to construct the response to the user prompt, our approach is to avoid using internal LLM factual knowledge altogether. Instead, verified RDF statements are compared to chunks of external documents retrieved through a web search or Wikipedia. To assess the possible application of this retrieval augmented generation (RAG) workflow on biosciences content, we evaluated 1,719 positive statements from the BioRED dataset and the same number of newly generated negative statements. The resulting precision is 88 %, and recall is 44 %. This indicates that the method requires human oversight. We also evaluated the method on the SNLI dataset which allowed us to compare our approach with models specifically tuned for the natural language inference task. We demonstrate the method on Wikidata, where a SPARQL query is used to automatically retrieve statements needing verification. Overall, the results suggest that LLMs could be used for large-scale verification of statements in KGs, a task previously unfeasible due to human annotation costs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
(2013) · Cited by 5,726
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377
DBpedia - A crystallization point for the Web of Data
Bizer, C., Lehmann, J., Kobilarov, G., Auer, S., Becker, C., Cyganiak, R., & Hellmann, S. (2009) · Journal of Web Semantics · doi:10.1016/j.websem.2009.07.002 · Cited by 2,137
A Decomposable Attention Model for Natural Language Inference
Parikh, A., Täckström, O., Das, D., & Uszkoreit, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1244 · Cited by 1,296
Observed versus latent features for knowledge base and text inference
Toutanova, K., & Chen, D. (2015) · Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality · doi:10.18653/v1/w15-4007 · Cited by 1,070
PubTator central: automated concept annotation for biomedical full text articles
Wei, C.-H., Allot, A., Leaman, R., & Lu, Z. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz389 · Cited by 400
Shefchek, K. A., Harris, N. L., Gargano, M., Matentzoglu, N., Unni, D., Brush, M., Keith, D., Conlin, T., Vasilevsky, N., Zhang, X. A., et al. (2020) · Nucleic Acids Research · doi:10.1093/nar/gkz997 · Cited by 236
BioRED: a rich biomedical relation extraction dataset
Luo, L., Lai, P.-T., Wei, C.-H., Arighi, C. N., & Lu, Z. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac282 · Cited by 159
Proceedings of the 33rd Annual ACM Symposium on Applied Computing
van Roosmalen, J., Vranken, H. P. E., van Eekelen, R. N., Haddad, H. M., Wainwright, R. L., & Chbeir, R. (2018) · doi:10.1145/3167132 · Cited by 154
Research works printed but not linked: 8
- Ding, Z., Huang, W., Liang, J., Yang, D., & Xiao, Y. (2024). Improving recall of large language models: A model collaboration approach for relational triple extraction. arXiv preprint arXiv:2404.09593. · arXiv
- Efeoglu, S., & Paschke, A. (2024). Relation extraction with fine-tuned large language models in retrieval augmented generation frameworks. arXiv preprint arXiv:2406.14745. · arXiv
- Han, R., Peng, T., Yang, C., Wang, B., Liu, L., & Wan, X. (2023). Is information extraction solved by ChatGPT? An analysis of performance, evaluation criteria, robustness and errors. arXiv preprint arXiv:2305.14450. · arXiv
- Lee, S., Wang, Z.J., Chakravarthy, A., Helbling, A., Peng, S., Phute, M., Chau, D.H., & Kahng, M. (2024). LLM attributor: Interactive visual attribution for LLM generation. arXiv preprint arXiv:2404.01361. · DOI
- Reshetiloff, 2001, Chesapeake Bay night-lights add sparkle to woods, water, Bay Journal
- Wei, X., Cui, X., Cheng, N., Wang, X., Zhang, X., Huang, S., Xie, P., Xu, J., Chen, Y., Zhang, M., Jiang, Y., & Han, W. (2023). Zero-shot information extraction via chatting with ChatGPT. arXiv preprint arXiv:2302.10205. · arXiv
- Wu, K., Wu, E., Cassasola, A., Zhang, A., Wei, K., Nguyen, T., Riantawan, S., Riantawan, P.S., Ho, D.E., Zou, J., & Zou, J. (2024). How well do LLMs cite relevant medical references? An evaluation framework and analyses. arXiv preprint arXiv:2402.02008. · arXiv
- 2024, Wikidata, In Wikipedia, The Free Encyclopedia
Metadata
- Authors
- Adam, D., & Kliegr, T.
- Venue
- Information Processing & Management
- Publisher
- Elsevier BV
- Published
- 2025-07
- Volume
- 62
- Issue
- 4
- Pages
- 104128
- ISSN
- 0306-4573
- Cited by
- 23
- References
- 21
- Language
- en
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- doi.org/10.15223/policy-017 (stm-asf)
- doi.org/10.15223/policy-037 (stm-asf)
- doi.org/10.15223/policy-012 (stm-asf)
- doi.org/10.15223/policy-029 (stm-asf)
- doi.org/10.15223/policy-004 (stm-asf)
- Funders
- Prague University of Economics and Business
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Traceable LLM-based validation of statements in knowledge graphs. https://app.cochat.ai/papers/10.1016/j.ipm.2025.104128
- The paper
- Adam, D., & Kliegr, T. (2025). Traceable LLM-based validation of statements in knowledge graphs. Information Processing & Management. https://doi.org/10.1016/j.ipm.2025.104128
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.