Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index; none of the 4 sources its claims cite is checked for support yet.

Generate FAIR Literature Surveys with Scholarly Knowledge Graphs

Open access
Oelen, Allard, Jaradeh, Mohamad Yaser, Stocker, Markus, Auer, Sören · 2020 · Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020

Abstract

Reviewing scientific literature is a cumbersome, time consuming but crucial activity in research. Leveraging a scholarly knowledge graph, we present a methodology and a system for comparing scholarly literature, in particular research contributions describing the addressed problem, utilized materials, employed methods and yielded results. The system can be used by researchers to quickly get familiar with existing work in a specific research domain (e.g., a concrete research question or hypothesis). Additionally, it can be used to publish literature surveys following the FAIR Data Principles. The methodology to create a research contribution comparison consists of multiple tasks, specifically: (a) finding similar contributions, (b) aligning contribution descriptions, (c) visualizing and finally (d) publishing the comparison. The methodology is implemented within the Open Research Knowledge Graph (ORKG), a scholarly infrastructure that enables researchers to collaboratively describe, find and compare research contributions. We evaluate the implementation using data extracted from published review articles. The evaluation also addresses the FAIRness of comparisons published with the ORKG.

Overview

CoChat Summary

TL;DR

The paper presents a methodology and ORKG-based system for generating and publishing FAIR literature surveys by comparing structured research contributions from a scholarly knowledge graph. Evaluation shows original survey tables can be recreated without information loss, published comparisons rank high in FAIRness, and the ORKG alignment approach outperforms a naive comparison method.

Key findings

  • Using data imported from four review papers (12 tables, 169 papers, 3,750 statements), the ORKG comparison service could recreate the same tabular views as originally published, with no information loss reported for the imported tables.
  • Published comparisons rank high in FAIRness: the service provides unique identifiers, machine- and user-generated metadata, HTTP access without authentication, separate metadata storage, RDF/CSV exports, and a CC BY-SA license, though interoperability and reusability still require user effort.
  • The ORKG approach outperforms a naive property-alignment approach; for 8 compared contributions, Table 3 reports 14,421 seconds for the naive approach versus 0.0204 seconds for ORKG.

Methods

  • Design: a five-step comparison methodology implemented in the Open Research Knowledge Graph (ORKG): finding comparison candidates, selecting related statements, aligning contribution descriptions, visualizing comparisons, and publishing FAIR comparisons.
  • Data/sample: the ORKG was populated by importing tabular survey data from four review papers, yielding 12 tables, 169 reviewed papers, and 3,750 statements; evaluation compared original tables with ORKG-generated tables, assessed FAIR sub-principles, and benchmarked performance.
  • Main analysis: similar contributions are found with TF-IDF; related statements are selected transitively to depth δ = 5; properties are aligned using pre-trained fastText embeddings with similarity threshold τ ≥ 0.9; visualization displays properties shared by at least α ≥ 2 contributions by default; performance compared naive versus ORKG alignment for 2–8 contributions with on average ten properties per paper.

Limitations

  • Stated in the paper: the ORKG does not yet contain sufficient contribution descriptions for the comparison functionality to be practically useful for researchers, and more contribution data is needed to evaluate components such as finding related papers or aligning similar properties.
  • Stated in the paper: the information representation evaluation does not give insight into usefulness or usability; FAIR is not a standard and its principles are permissive and not prescriptive, so implementation and evaluation are subject to interpretation; interoperability and reusability can be improved, users are responsible for correct metadata, and there is currently no community standard to describe comparison data.
  • Evident from the text: the performance evaluation is limited to eight papers because the naive approach does not scale, and the imported evaluation data comes from only four review papers and 12 tables; no user study is reported.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3383583.3398520

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Research contribution comparison methodology.
Figure 1 Research contribution comparison methodology.CC-BY-SA · From the PDF
Figure 2 Implementation of the first step of the methodology: the selection of comparison candidates. Showing both the similarity-based and the manual selection approaches.
Figure 2 Implementation of the first step of the methodology: the selection of comparison candidates. Showing both the similarity-based and the manual selection approaches.CC-BY-SA · From the PDF
Figure 3 Box showing the manually selected contributions.
Figure 3 Box showing the manually selected contributions.CC-BY-SA · From the PDF
Figure 4 Comparison of research contributions related to visualization tools.
Figure 4 Comparison of research contributions related to visualization tools.CC-BY-SA · From the PDF
Figure 5 The graph structure of the metadata for a published comparison. The dcterms: prefix denotes the Dublin Core Metadata Terms ontology.
Figure 5 The graph structure of the metadata for a published comparison. The dcterms: prefix denotes the Dublin Core Metadata Terms ontology.CC-BY-SA · From the PDF
Figure 6 Partial graph structure of an imported paper. Orange colored resources indicate potentially interesting values for a paper comparison.
Figure 6 Partial graph structure of an imported paper. Orange colored resources indicate potentially interesting values for a paper comparison.CC-BY-SA · From the PDF

Verification

Claims the paper makes

0 Valid · 13 Not valid · 3 Not checked · 63 Own results and interpretations

21 more claims

Create a free account to see all 79 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 17
  • Probably not a substantial claim. 49

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 52 references

Cited by

See all 52

Metadata

Authors
Oelen, A., Jaradeh, M. Y., Stocker, M., & Auer, S.
Affiliations
  • Leibniz University Hannover (DE)
  • Technische Informationsbibliothek (TIB) (DE)
Venue
Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020
Publisher
ACM
Published
2020-08
Pages
97-106
ISSN
1552-5996 (ISSN-L)
OpenAlex
W3033889636
Cited by
52
References
52
Citations by year
2026: 3 · 2025: 7 · 2024: 11 · 2023: 7 · 2022: 9 · 2021: 10 · 2020: 5
FWCI
2.05
Type
conference paper
Language
en
Open access
Locations
  • ACM/IEEE Joint Conference on Digital Libraries (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-sa)
  • RENATE (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
  • RENATE
Funders
H2020 European Research Council (819536)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
  • Social Sciences › Arts and Humanities › Literature and Literary Theory › Digital Humanities and Scholarship
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Keywords
scholarly knowledge graphs, literature survey, FAIR principles, Open Research Knowledge Graph, similarity search, knowledge graph alignment, literature visualization, fairness evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Generate FAIR Literature Surveys with Scholarly Knowledge Graphs. https://app.cochat.ai/papers/10.1145/3383583.3398520
The paper
Oelen, A., Jaradeh, M. Y., Stocker, M., & Auer, S. (2020). Generate FAIR Literature Surveys with Scholarly Knowledge Graphs. Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020. https://doi.org/10.1145/3383583.3398520

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.