Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 66 of the 67 references it cites were found in a registry or an index; none of the 27 sources its claims cite is checked for support yet.

Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs

Open access
Negro, Alessandro, Montagna, Fabio, Teng, Michael N., Neal, Tempestt, Thomas, Sylvia, King, Sayde, Khan, Ridita · 2023 · Frontiers in Research Metrics and Analytics

Abstract

The COVID-19 pandemic highlighted two critical barriers hindering rapid response to novel pathogens. These include inefficient use of existing biological knowledge about treatments, compounds, gene interactions, proteins, etc. to fight new diseases, and the lack of assimilation and analysis of the fast-growing knowledge about new diseases to quickly develop new treatments, vaccines, and compounds. Overcoming these critical challenges has the potential to revolutionize global preparedness for future pandemics. Accordingly, this article introduces a novel knowledge graph application that functions as both a repository of life science knowledge and an analytics platform capable of extracting time-sensitive insights to uncover evolving disease dynamics and, importantly, researchers' evolving understanding. Specifically, we demonstrate how to extract time-bounded key concepts, also leveraging existing ontologies, from evolving scholarly articles to create a single temporal connected source of truth specifically related to COVID-19. By doing so, current knowledge can be promptly accessed by both humans and machines, from which further understanding of disease outbreaks can be derived. We present key findings from the temporal analysis, applied to a subset of the resulting knowledge graph known as the temporal keywords knowledge graph, and delve into the detailed capabilities provided by this innovative approach.

Overview

CoChat Summary

TL;DR

The paper introduces a temporal knowledge graph and a TagGraph application built from COVID-19 literature and public biomedical sources, using monthly keyword co-occurrence snapshots and Role-Dynamics to trace evolving research topics. It reports interpretable roles and neighborhood patterns—such as machine learning rising and hydroxychloroquine declining—and a clinician/researcher survey motivating automated trend detection.

Key findings

  • Temporal analysis of the TagGraph from March 2020 to March 2022 identified five roles; Role 0 and Role 2 correlate with centrality/importance (Role 0: PageRank 0.843, closeness 0.861, degree 0.866; Role 2: PageRank 0.691, closeness 0.637, degree 0.808, local clustering −0.823), Role 1 appears unrelated/noisy, and Roles 3 and 4 represent peripheral or isolated nodes.
  • Case examples show “machine learning” had a steady progression in roles 2 and 0 over time, while “hydroxychloroquine” initially spiked in roles 0 and 2 and then degraded, with role 3 increasing.
  • Top SARS-CoV-2 neighbor tags included ACE2, serology, MERS-CoV, spike protein, and transmission; top COVID-19 neighbor tags included mortality, social distancing, healthcare workers, lockdown, anxiety, and mental health.
  • A survey of 26 clinicians and research scientists found 67% cited transmission/asymptomatic spread as information they wished they knew earlier, 54% used sources such as PubMed, news, CDC, and IDSA, and 30% had no seminal source; coding had bi-rater agreement of 0.72 and consensus score of 32%.

Methods

  • The authors constructed a knowledge graph from public sources including Hetionet, Uniprot, CORD-19, Drug Repurposing Knowledge Graph, Gene Ontology, and MeSH, using the Linked Data lifecycle and GraphAware Hume, with NLP named entity recognition and entity relationship extraction.
  • They built a TagGraph from author keywords, clustered synonymous tags using SPECTER embeddings and DBSCAN, computed monthly co-occurrence snapshots with association strength SA = c_ij / (s_i s_j), and applied ReFeX structural features plus RolX matrix factorization via Role-Dynamics.
  • A qualitative survey with 26 participants (physicians, nurses, other health professionals, and Ph.D.-level researchers) used five open-response questions and thematic coding by three researchers under USF IRB Study #01211.

Limitations

  • Stated: the TagGraph is only a small portion of a larger knowledge graph; tag cleanup and merging could improve with alternative clustering algorithms, ontologies, taxonomies, and dictionaries; roles are difficult to interpret; the analysis considered only directly connected nodes, postponing egonet analysis; and unexplored sources include patents, social media, news, country regulations, and WHO/healthcare communication.
  • Stated/evident: survey coding agreement was limited, with bi-rater agreement of 0.72 and consensus score of 32%, attributed to 70 available codes and up to six codes per response.
  • Evident from the text: the temporal analysis is bounded to March 2020–March 2022 and monthly snapshots, and results depend on author-provided keywords and the selected public data sources.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3389/frma.2023.1204801

Cites

66 of 66 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 A portion of the knowledge graph schema. This schema captures data about genes, diseases, compounds, and side effects, along with their interactions, e.g., how a disease is connected to a specific gene, how it can be treated by a specific compound, and the side effects of such compound. Research manuscripts are connected from one author to another author by institution, and relevant relationships between manuscript sections are retained.
Figure 1 A portion of the knowledge graph schema. This schema captures data about genes, diseases, compounds, and side effects, along with their interactions, e.g., how a disease is connected to a specific gene, how it can be treated by a specific compound, and the side effects of such compound. Research manuscripts are connected from one author to another author by institution, and relevant relationships between manuscript sections are retained.CC BY · PubMed Central
Figure 2 A portion of the projected graph, where the OCCURS_WITH relationship connects keys that have been mentioned in the same paper. The highlighted relationship shows that the keys “SARSCoV-2” and “serology” appear often together, although with varying frequencies over the period under consideration. As a result, each relationship is associated with a weight that is a function of time.
Figure 2 A portion of the projected graph, where the OCCURS_WITH relationship connects keys that have been mentioned in the same paper. The highlighted relationship shows that the keys “SARSCoV-2” and “serology” appear often together, although with varying frequencies over the period under consideration. As a result, each relationship is associated with a weight that is a function of time.CC BY · PubMed Central
Figure 3 Flowchart for the temporal graph analysis. From the heterogeneous graph that represents the initial knowledge graph, monthly snapshots are extracted, describing the co-occurrence of keywords in papers published each month. The REFEX feature extraction algorithm is applied to each snapshot, associating each keyword with a different feature vector in each snapshot. The features are then aggregated and processed using the RolX algorithm, which assigns a role to each keyword for each month.
Figure 3 Flowchart for the temporal graph analysis. From the heterogeneous graph that represents the initial knowledge graph, monthly snapshots are extracted, describing the co-occurrence of keywords in papers published each month. The REFEX feature extraction algorithm is applied to each snapshot, associating each keyword with a different feature vector in each snapshot. The features are then aggregated and processed using the RolX algorithm, which assigns a role to each keyword for each month.CC BY · PubMed Central
Figure 4 Conversion of each node into a vector representing the node's topological feature at different scales using ReFeX (Henderson et al., 2011).
Figure 4 Conversion of each node into a vector representing the node's topological feature at different scales using ReFeX (Henderson et al., 2011).CC BY · PubMed Central
Figure 5 Compression of ReFeX feature vectors into smaller role vectors using RolX.
Figure 5 Compression of ReFeX feature vectors into smaller role vectors using RolX.CC BY · PubMed Central
Figure 6 An example of a graph containing nodes with different role distribution. Role 0 and Role 1 nodes are strongly connected and central, Role 3 nodes are almost disconnected from the central nodes, and Role 4 nodes are structured such that they are peripheral but connected.
Figure 6 An example of a graph containing nodes with different role distribution. Role 0 and Role 1 nodes are strongly connected and central, Role 3 nodes are almost disconnected from the central nodes, and Role 4 nodes are structured such that they are peripheral but connected.CC BY · PubMed Central
Figure 7 Role evolution comparison: the “machine learning” keyword quickly transitions from a marginal role to a relatively central one between May and June 2020, with a consistently positive trend that makes it a highly relevant keyword. On the other hand, the “Hydroxychloroquine” keyword displays fluctuating patterns that reflect the scientific community's interest in this molecule, with periods of higher and lower interest.
Figure 7 Role evolution comparison: the “machine learning” keyword quickly transitions from a marginal role to a relatively central one between May and June 2020, with a consistently positive trend that makes it a highly relevant keyword. On the other hand, the “Hydroxychloroquine” keyword displays fluctuating patterns that reflect the scientific community's interest in this molecule, with periods of higher and lower interest.CC BY · PubMed Central
Figure 8 Temporal trend of central keys characterized by high values associated with Role 0 and Role 2.
Figure 8 Temporal trend of central keys characterized by high values associated with Role 0 and Role 2.CC BY · PubMed Central

Verification

Claims the paper makes

0 Valid · 2 Not valid · 20 Not checked · 64 Own results and interpretations

26 more claims

Create a free account to see all 86 claims and to verify the 25 references still open.

See all claims — free account
  • Probably not a substantial claim. 50
  • It may state two separate claims as one. 17

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 67 references

Research works printed but not linked: 1

  1. Negro, 2023, Knowledge Graph Applied

Cited by

See all 6

Metadata

Authors
Negro, A., Montagna, F., Teng, M. N., Neal, T., Thomas, S., King, S., & Khan, R.
Affiliations
University of South Florida (US)
Corresponding author
  • Negro, Alessandro
  • Neal, Tempestt
Venue
Frontiers in Research Metrics and Analytics
Published
2023-08-03
Volume
8
Pages
1204801-1204801
ISSN
2504-0537 (ISSN-L)
OpenAlex
W4385519666
Cited by
6
References
67
Citations by year
2026: 2 · 2025: 1 · 2024: 2 · 2023: 1
FWCI
0.22
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Research Metrics and Analytics (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Science Foundation (2028051)
Topics
  • Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
  • Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
  • Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
Keywords
COVID-19 pandemic, temporal knowledge graphs, knowledge graph evolution, ontology (information science), pandemic preparedness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs. https://app.cochat.ai/papers/10.3389/frma.2023.1204801
The paper
Negro, A., Montagna, F., Teng, M. N., Neal, T., Thomas, S., King, S., & Khan, R. (2023). Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs. Frontiers in Research Metrics and Analytics. https://doi.org/10.3389/frma.2023.1204801

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.