Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 8

The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.

ProVe: A pipeline for automated provenance verification of knowledge graphs against textual sources

Amaral, Gabriel, Rodrigues, Odinaldo, Simperl, Elena·2023·Semantic Web
DOI 10.3233/sw-233467
Open this paper in CoChat — free account

Abstract

Knowledge Graphs are repositories of information that gather data from a multitude of domains and sources in the form of semantic triples, serving as a source of structured data for various crucial applications in the modern web landscape, from Wikipedia infoboxes to search engines. Such graphs mainly serve as secondary sources of information and depend on well-documented and verifiable provenance to ensure their trustworthiness and usability. However, their ability to systematically assess and assure the quality of this provenance, most crucially whether it properly supports the graph’s information, relies mainly on manual processes that do not scale with size. ProVe aims at remedying this, consisting of a pipelined approach that automatically verifies whether a Knowledge Graph triple is supported by text extracted from its documented provenance. ProVe is intended to assist information curators and consists of four main steps involving rule-based methods and machine learning models: text extraction, triple verbalisation, sentence selection, and claim verification. ProVe is evaluated on a Wikidata dataset, achieving promising results overall and excellent performance on the binary classification task of detecting support from provenance, with 87.5 % accuracy and 82.9 % F1-macro on text-rich sources. The evaluation data and scripts used in this paper are available in GitHub and Figshare.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3233/sw-233467

Cites

59 of 59 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 8

  1. M. Acosta, A. Zaveri, E. Simperl, D. Kontokostas, S. Auer and J. Lehmann, Crowdsourcing linked data quality assessment, in: The Semantic Web – ISWC 2013, H. Alani, L. Kagal, A. Fokoue, P. Groth, C. Biemann, J.X. Parreira, L. Aroyo, N. Noy, C. Welty and K. Janowicz, eds, Springer, Berlin, Heidelberg, 2013, pp. 260–276. ISBN 978-3-642-41338-4.
  2. A. Ammar and R. Celebi, Fact validation with knowledge graph embeddings, in: 2019 ISWC Satellite Tracks (Posters and Demonstrations, Industry, and Outrageous Ideas), ISWC 2019-Satellites. CEUR-WSceurws, 2019, ISSN 1613-0073. http://hdl.handle.net/10754/679467.
  3. G. Flouris, Y. Roussakis, M. Poveda-Villalón, P.N. Mendes and I. Fundulaki, Using provenance for quality assessment and repair in linked open data, in: Joint Workshop on Knowledge Evolution and Ontology Dynamics, 2012, oeg. https://oa.upm.es/14477/.
  4. P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel and D. Kiela, Retrieval-augmented generation for knowledge-intensive NLP tasks, in: Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS’20, Curran Associates Inc., Red Hook, NY, USA, 2020. ISBN 9781713829546.
  5. I. Loshchilov and F. Hutter, Decoupled weight decay regularization, in: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6–9, 2019, OpenReview.net, 2019. https://openreview.net/forum?id=Bkg6RiCqY7.
  6. A. Sathe, S. Ather, T.M. Le, N. Perry and J. Park, Automated fact-checking of claims from Wikipedia, in: Proceedings of the Twelfth Language Resources and Evaluation Conference, European Language Resources Association, Marseille, France, 2020, pp. 6874–6882. ISBN 979-10-95546-34-4. https://aclanthology.org/2020.lrec-1.849.
  7. 10.1007/978-3-030-45442-5_45 · DOI
  8. J. Thorne and A. Vlachos, Automated fact checking: Task formulations, methods and future directions, in: Proceedings of the 27th International Conference on Computational Linguistics, Association for Computational Linguistics, Santa Fe, New Mexico, USA, 2018, pp. 3346–3359, https://aclanthology.org/C18-1283.

Cited by

See all 8

Metadata

Authors
Amaral, G., Rodrigues, O., & Simperl, E.
Venue
Semantic Web
Publisher
SAGE Publications
Published
2023-09-12
Pages
1-34
ISSN
  • 2210-4968
  • 1570-0844
DOI
10.3233/sw-233467
Cited by
8
References
59
Licence
creativecommons.org/licenses/by/4.0/ (unspecified)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.