Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 23

The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.

Traceable LLM-based validation of statements in knowledge graphs

Adam, Daniel, Kliegr, Tomáš · 2025 · Information Processing & Management

Abstract

This article presents a method for verifying RDF triples using LLMs , with an emphasis on providing traceable arguments. Because the LLMs cannot currently reliably identify the origin of the information used to construct the response to the user prompt, our approach is to avoid using internal LLM factual knowledge altogether. Instead, verified RDF statements are compared to chunks of external documents retrieved through a web search or Wikipedia. To assess the possible application of this retrieval augmented generation (RAG) workflow on biosciences content, we evaluated 1,719 positive statements from the BioRED dataset and the same number of newly generated negative statements. The resulting precision is 88 %, and recall is 44 %. This indicates that the method requires human oversight. We also evaluated the method on the SNLI dataset which allowed us to compare our approach with models specifically tuned for the natural language inference task. We demonstrate the method on Wikidata, where a SPARQL query is used to automatically retrieve statements needing verification. Overall, the results suggest that LLMs could be used for large-scale verification of statements in KGs, a task previously unfeasible due to human annotation costs.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.ipm.2025.104128

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 21 references

Research works printed but not linked: 8

  1. Ding, Z., Huang, W., Liang, J., Yang, D., & Xiao, Y. (2024). Improving recall of large language models: A model collaboration approach for relational triple extraction. arXiv preprint arXiv:2404.09593. · arXiv
  2. Efeoglu, S., & Paschke, A. (2024). Relation extraction with fine-tuned large language models in retrieval augmented generation frameworks. arXiv preprint arXiv:2406.14745. · arXiv
  3. Han, R., Peng, T., Yang, C., Wang, B., Liu, L., & Wan, X. (2023). Is information extraction solved by ChatGPT? An analysis of performance, evaluation criteria, robustness and errors. arXiv preprint arXiv:2305.14450. · arXiv
  4. Lee, S., Wang, Z.J., Chakravarthy, A., Helbling, A., Peng, S., Phute, M., Chau, D.H., & Kahng, M. (2024). LLM attributor: Interactive visual attribution for LLM generation. arXiv preprint arXiv:2404.01361. · DOI
  5. Reshetiloff, 2001, Chesapeake Bay night-lights add sparkle to woods, water, Bay Journal
  6. Wei, X., Cui, X., Cheng, N., Wang, X., Zhang, X., Huang, S., Xie, P., Xu, J., Chen, Y., Zhang, M., Jiang, Y., & Han, W. (2023). Zero-shot information extraction via chatting with ChatGPT. arXiv preprint arXiv:2302.10205. · arXiv
  7. Wu, K., Wu, E., Cassasola, A., Zhang, A., Wei, K., Nguyen, T., Riantawan, S., Riantawan, P.S., Ho, D.E., Zou, J., & Zou, J. (2024). How well do LLMs cite relevant medical references? An evaluation framework and analyses. arXiv preprint arXiv:2402.02008. · arXiv
  8. 2024, Wikidata, In Wikipedia, The Free Encyclopedia

Metadata

Authors
Adam, D., & Kliegr, T.
Venue
Information Processing & Management
Publisher
Elsevier BV
Published
2025-07
Volume
62
Issue
4
Pages
104128
ISSN
0306-4573
Cited by
23
References
21
Language
en
Funders
Prague University of Economics and Business

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Traceable LLM-based validation of statements in knowledge graphs. https://app.cochat.ai/papers/10.1016/j.ipm.2025.104128
The paper
Adam, D., & Kliegr, T. (2025). Traceable LLM-based validation of statements in knowledge graphs. Information Processing & Management. https://doi.org/10.1016/j.ipm.2025.104128

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.