The DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index; none of the 4 sources its claims cite is checked for support yet.
Generate FAIR Literature Surveys with Scholarly Knowledge Graphs
Open accessAbstract
Reviewing scientific literature is a cumbersome, time consuming but crucial activity in research. Leveraging a scholarly knowledge graph, we present a methodology and a system for comparing scholarly literature, in particular research contributions describing the addressed problem, utilized materials, employed methods and yielded results. The system can be used by researchers to quickly get familiar with existing work in a specific research domain (e.g., a concrete research question or hypothesis). Additionally, it can be used to publish literature surveys following the FAIR Data Principles. The methodology to create a research contribution comparison consists of multiple tasks, specifically: (a) finding similar contributions, (b) aligning contribution descriptions, (c) visualizing and finally (d) publishing the comparison. The methodology is implemented within the Open Research Knowledge Graph (ORKG), a scholarly infrastructure that enables researchers to collaboratively describe, find and compare research contributions. We evaluate the implementation using data extracted from published review articles. The evaluation also addresses the FAIRness of comparisons published with the ORKG.
Overview
CoChat Summary
TL;DR
The paper presents a methodology and ORKG-based system for generating and publishing FAIR literature surveys by comparing structured research contributions from a scholarly knowledge graph. Evaluation shows original survey tables can be recreated without information loss, published comparisons rank high in FAIRness, and the ORKG alignment approach outperforms a naive comparison method.
Key findings
- Using data imported from four review papers (12 tables, 169 papers, 3,750 statements), the ORKG comparison service could recreate the same tabular views as originally published, with no information loss reported for the imported tables.
- Published comparisons rank high in FAIRness: the service provides unique identifiers, machine- and user-generated metadata, HTTP access without authentication, separate metadata storage, RDF/CSV exports, and a CC BY-SA license, though interoperability and reusability still require user effort.
- The ORKG approach outperforms a naive property-alignment approach; for 8 compared contributions, Table 3 reports 14,421 seconds for the naive approach versus 0.0204 seconds for ORKG.
Methods
- Design: a five-step comparison methodology implemented in the Open Research Knowledge Graph (ORKG): finding comparison candidates, selecting related statements, aligning contribution descriptions, visualizing comparisons, and publishing FAIR comparisons.
- Data/sample: the ORKG was populated by importing tabular survey data from four review papers, yielding 12 tables, 169 reviewed papers, and 3,750 statements; evaluation compared original tables with ORKG-generated tables, assessed FAIR sub-principles, and benchmarked performance.
- Main analysis: similar contributions are found with TF-IDF; related statements are selected transitively to depth δ = 5; properties are aligned using pre-trained fastText embeddings with similarity threshold τ ≥ 0.9; visualization displays properties shared by at least α ≥ 2 contributions by default; performance compared naive versus ORKG alignment for 2–8 contributions with on average ten properties per paper.
Limitations
- Stated in the paper: the ORKG does not yet contain sufficient contribution descriptions for the comparison functionality to be practically useful for researchers, and more contribution data is needed to evaluate components such as finding related papers or aligning similar properties.
- Stated in the paper: the information representation evaluation does not give insight into usefulness or usability; FAIR is not a standard and its principles are permissive and not prescriptive, so implementation and evaluation are subject to interpretation; interoperability and reusability can be improved, users are responsible for correct metadata, and there is currently no community standard to describe comparison data.
- Evident from the text: the performance evaluation is limited to eight papers because the naive approach does not scale, and the imported evaluation data comes from only four review papers and 12 tables; no user study is reported.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
0 Valid · 13 Not valid · 3 Not checked · 63 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Data is (partially) described using the ORKG core ontology and other ontologies we use to canonicalize the representation of relevant information content types.”
§ Generate FAIR Literature Surveys with Scholarly Knowledge Graphs > 6.2 FAIR data evaluation
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the naive approach, the”
§ Generate FAIR Literature Surveys with Scholarly Knowledge Graphs > 6.3 Performance evaluation
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This methodology has been used to populate the ORKG with comparable paper data.”
§ Generate FAIR Literature Surveys with Scholarly Knowledge Graphs > 5.1 Leverage legacy review paper tables
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Otherwise, findability, accessibility and to some extent also interoperability are largely handled by the service.”
§ Generate FAIR Literature Surveys with Scholarly Knowledge Graphs > 6.2 FAIR data evaluation
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The evaluation of the FAIR principles shows that comparisons published with our service rank high in FAIRness, which can be even further increased with some effort from users.”
§ Generate FAIR Literature Surveys with Scholarly Knowledge Graphs > 6.2 FAIR data evaluation
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
21 more claims
Create a free account to see all 79 claims and every source verdict.
- It may state two separate claims as one. 17
- Probably not a substantial claim. 49
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The Open Research Knowledge Graph (ORKG) (Jaradeh, et al., 2019) addresses this problem by representing scholarly knowledge in a machine actionable, structured, and semantic manner, and providing services for comparing scholarly knowledge (Oelen et al., 2020) in tabular and other visual forms (Wiens et al., 2020) as well as other types of services for efficient reuse of scholarly knowledge.”
From Persistent Identification and Interlinking of FAIR Scholarly Knowledge (2022) · cites this paper as Oelen et al., 2020
Citations
Cites
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423
Binary Codes capable of currecting deletions, insertions, and reversals
Levenshtein, V. I. (1965) · Soviet physics. Doklady · Cited by 14,807
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
Analyzing the Past to Prepare for the Future: Writing a Literature Review
Webster, J., & Watson, R. T. (2002) · MIS Quarterly · doi:10.2307/4132319 · Cited by 7,449
Educational Research: An Introduction
Bennett, N., Borg, W. R., & Gall, M. D. (1984) · British Journal of Educational Studies · doi:10.2307/3121583 · Cited by 5,595
Writing Integrative Literature Reviews: Guidelines and Examples
Torraco, R. J. (2005) · Human Resource Development Review · doi:10.1177/1534484305278283 · Cited by 3,534
(1961) · Journal of the Optical Society of America · doi:10.1364/josa.51.000248 · Cited by 2,288
Mitzias, P., Riga, M., Waddington, S. N., Kontopoulos, E., Μεδίτσκος, Γ., Laurenson, P., & Kompatsiaris, I. Y. (2015) · Cited by 1,240
Ontology Matching: State of the Art and Future Challenges
Shvaiko, P., & Euzenat, J. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.253 · Cited by 1,148
String Comparator Metrics and Enhanced Decision Rules in the Fellegi-Sunter Model of Record Linkage.
Winkler, W. E. (1990) · Cited by 787
Cited by
Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases
Weikum, G., Dong, X. L., Razniewski, S., & Suchanek, F. M. (2021) · Foundations and Trends in Databases · doi:10.1561/1900000064 · Cited by 202
Improving Access to Scientific Literature with Knowledge Graphs
Auer, S., Oelen, A., Haris, M., Stocker, M., D’Souza, J., Farfar, K. E., Vogt, L., Prinz, M., Wiens, V., & Jaradeh, M. Y. (2020) · Bibliothek Forschung und Praxis · doi:10.1515/bfp-2020-2042 · Cited by 82
Self-citationScholarly knowledge graphs through structuring scholarly communication: a review
Verma, S., Bhatia, R., Harit, S., & Batish, S. (2023) · Complex & Intelligent Systems · doi:10.1007/s40747-022-00806-6 · Cited by 78
Azher, I. A., Seethi, V. D. R., Akella, A. P., & Alhoori, H. (2024) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3677389.3702605 · Cited by 73
Brack, A., Hoppe, A., Stocker, M., Auer, S., & Ewerth, R. (2021) · International Journal on Digital Libraries · doi:10.1007/s00799-021-00306-x · Cited by 37
Self-citation
Metadata
- Authors
- Oelen, A., Jaradeh, M. Y., Stocker, M., & Auer, S.
- Affiliations
- Leibniz University Hannover (DE)
- Technische Informationsbibliothek (TIB) (DE)
- Venue
- Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020
- Publisher
- ACM
- Published
- 2020-08
- Pages
- 97-106
- ISSN
- 1552-5996 (ISSN-L)
- OpenAlex
- W3033889636
- Cited by
- 52
- References
- 52
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 11 · 2023: 7 · 2022: 9 · 2021: 10 · 2020: 5
- FWCI
- 2.05
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM/IEEE Joint Conference on Digital Libraries (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-sa)
- RENATE (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- RENATE
- Funders
- H2020 European Research Council (819536)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
- Social Sciences › Arts and Humanities › Literature and Literary Theory › Digital Humanities and Scholarship
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- scholarly knowledge graphs, literature survey, FAIR principles, Open Research Knowledge Graph, similarity search, knowledge graph alignment, literature visualization, fairness evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generate FAIR Literature Surveys with Scholarly Knowledge Graphs. https://app.cochat.ai/papers/10.1145/3383583.3398520
- The paper
- Oelen, A., Jaradeh, M. Y., Stocker, M., & Auer, S. (2020). Generate FAIR Literature Surveys with Scholarly Knowledge Graphs. Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020. https://doi.org/10.1145/3383583.3398520
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.