The DOI matches its Crossref record; 66 of the 67 references it cites were found in a registry or an index; none of the 27 sources its claims cite is checked for support yet.
Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs
Open accessAbstract
The COVID-19 pandemic highlighted two critical barriers hindering rapid response to novel pathogens. These include inefficient use of existing biological knowledge about treatments, compounds, gene interactions, proteins, etc. to fight new diseases, and the lack of assimilation and analysis of the fast-growing knowledge about new diseases to quickly develop new treatments, vaccines, and compounds. Overcoming these critical challenges has the potential to revolutionize global preparedness for future pandemics. Accordingly, this article introduces a novel knowledge graph application that functions as both a repository of life science knowledge and an analytics platform capable of extracting time-sensitive insights to uncover evolving disease dynamics and, importantly, researchers' evolving understanding. Specifically, we demonstrate how to extract time-bounded key concepts, also leveraging existing ontologies, from evolving scholarly articles to create a single temporal connected source of truth specifically related to COVID-19. By doing so, current knowledge can be promptly accessed by both humans and machines, from which further understanding of disease outbreaks can be derived. We present key findings from the temporal analysis, applied to a subset of the resulting knowledge graph known as the temporal keywords knowledge graph, and delve into the detailed capabilities provided by this innovative approach.
Overview
CoChat Summary
TL;DR
The paper introduces a temporal knowledge graph and a TagGraph application built from COVID-19 literature and public biomedical sources, using monthly keyword co-occurrence snapshots and Role-Dynamics to trace evolving research topics. It reports interpretable roles and neighborhood patterns—such as machine learning rising and hydroxychloroquine declining—and a clinician/researcher survey motivating automated trend detection.
Key findings
- Temporal analysis of the TagGraph from March 2020 to March 2022 identified five roles; Role 0 and Role 2 correlate with centrality/importance (Role 0: PageRank 0.843, closeness 0.861, degree 0.866; Role 2: PageRank 0.691, closeness 0.637, degree 0.808, local clustering −0.823), Role 1 appears unrelated/noisy, and Roles 3 and 4 represent peripheral or isolated nodes.
- Case examples show “machine learning” had a steady progression in roles 2 and 0 over time, while “hydroxychloroquine” initially spiked in roles 0 and 2 and then degraded, with role 3 increasing.
- Top SARS-CoV-2 neighbor tags included ACE2, serology, MERS-CoV, spike protein, and transmission; top COVID-19 neighbor tags included mortality, social distancing, healthcare workers, lockdown, anxiety, and mental health.
- A survey of 26 clinicians and research scientists found 67% cited transmission/asymptomatic spread as information they wished they knew earlier, 54% used sources such as PubMed, news, CDC, and IDSA, and 30% had no seminal source; coding had bi-rater agreement of 0.72 and consensus score of 32%.
Methods
- The authors constructed a knowledge graph from public sources including Hetionet, Uniprot, CORD-19, Drug Repurposing Knowledge Graph, Gene Ontology, and MeSH, using the Linked Data lifecycle and GraphAware Hume, with NLP named entity recognition and entity relationship extraction.
- They built a TagGraph from author keywords, clustered synonymous tags using SPECTER embeddings and DBSCAN, computed monthly co-occurrence snapshots with association strength SA = c_ij / (s_i s_j), and applied ReFeX structural features plus RolX matrix factorization via Role-Dynamics.
- A qualitative survey with 26 participants (physicians, nurses, other health professionals, and Ph.D.-level researchers) used five open-response questions and thematic coding by three researchers under USF IRB Study #01211.
Limitations
- Stated: the TagGraph is only a small portion of a larger knowledge graph; tag cleanup and merging could improve with alternative clustering algorithms, ontologies, taxonomies, and dictionaries; roles are difficult to interpret; the analysis considered only directly connected nodes, postponing egonet analysis; and unexplored sources include patents, social media, news, country regulations, and WHO/healthcare communication.
- Stated/evident: survey coding agreement was limited, with bi-rater agreement of 0.72 and consensus score of 32%, attributed to 70 available codes and up to six codes per response.
- Evident from the text: the temporal analysis is bounded to March 2020–March 2022 and monthly snapshots, and results depend on author-provided keywords and the selected public data sources.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
0 Valid · 2 Not valid · 20 Not checked · 64 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Role 0 and 2 shed light on aspects such as public and mental health, lockdown measures, and healthcare workers.”
§ Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs > 3. Results > 3.2. Story telling from temporal analysis of TagGraphs
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“High-level insights from our survey show that modes of transmission, particularly the infectivity of asymptomatic persons, were particularly concerning.”
§ Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs > 3. Results > 3.3. Establishing the critical need for KGs in pandemic response: a qualitative analysis of clinicians' resources and knowledge gathering of COVID-19
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In role 0, the same key terms appear at the top almost in all the time frames.”
§ Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs > 3. Results > 3.2. Story telling from temporal analysis of TagGraphs
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By utilizing signal analysis, certain tags exhibit a clear and strong signal for role 0 and role 2, which remains consistent throughout the entire history sampling, including “SARS-CoV-2”, “COVID-19”, and “Coronavirus”.”
§ Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs > 3. Results > 3.2. Story telling from temporal analysis of TagGraphs
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When analyzing the results of these tags, we noticed that they reflect noisy tags, i.e., tags that randomly appear in the network with no specific relevance of any type.”
§ Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs > 3. Results > 3.1. Role interpretations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
26 more claims
Create a free account to see all 86 claims and to verify the 25 references still open.
- Probably not a substantial claim. 50
- It may state two separate claims as one. 17
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs > 3. Results > 3.1. Role interpretations
“Role 0 and Role 1 nodes are strongly connected and central”
§ Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs > 3. Results > 3.1. Role interpretations
“they reflect noisy tags, i.e., tags that randomly appear in the network with no specific relevance of any type. The matrix factorization collected them under the same role 1.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs > 3. Results > 3.1. Role interpretations
“Role 0 and Role 2 are directly related to all the measures we extracted”
§ Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs > 3. Results > 3.1. Role interpretations
“| | Local Clustering | −0.823 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs > 3. Results > 3.2. Story telling from temporal analysis of TagGraphs
“For example, Table 2 shows the results for “SARS-CoV-2” across 3 months.”
§ Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs > 3. Results > 3.2. Story telling from temporal analysis of TagGraphs
“| 2020-04 | 2020-05 | 2020-06 | 2020-07 |”
Citations
Cites
(1996) · Cited by 20,179
Dexamethasone in Hospitalized Patients with Covid-19
The RECOVERY Collaborative Group (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2021436 · Cited by 11,477
COVID-19: consider cytokine storm syndromes and immunosuppression
Mehta, P., McAuley, D. F., Brown, M., Sanchez, E., Tattersall, R. S., & Manson, J. J. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30628-0 · Cited by 9,664
Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation
Wrapp, D., Wang, N., Corbett, K. S., Goldsmith, J. A., Hsieh, C.-L., Abiona, O., Graham, B. S., & McLellan, J. S. (2020) · Science · doi:10.1126/science.abb2507 · Cited by 8,897
UniProt: the Universal Protein Knowledgebase in 2023
The UniProt Consortium, Bateman, A., Martin, M.-J., Orchard, S., Magrane, M., Ahmad, S., Alpi, E., Bowler-Barnett, E. H., Britto, R., Bye-A-Jee, H., et al. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkac1052 · Cited by 7,210
Centers for Disease Control and Prevention
(2007) · Encyclopedia of Measurement and Statistics · doi:10.4135/9781412952644.n74 · Cited by 6,444
The proximal origin of SARS-CoV-2
Andersen, K. G., Rambaut, A., Lipkin, W. I., Holmes, E. C., & Garry, R. F. (2020) · Nature Medicine · doi:10.1038/s41591-020-0820-9 · Cited by 5,081
The Gene Ontology (GO) database and informatics resource
Gene Ontology Consortium (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh036 · Cited by 4,579
Temporal dynamics in viral shedding and transmissibility of COVID-19
He, X., Lau, E. H. Y., Wu, P., Deng, X., Wang, J., Hao, X., Lau, Y. C., Wong, J. Y., Guan, Y., Tan, X., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0869-5 · Cited by 4,247
Structural basis of receptor recognition by SARS-CoV-2
Shang, J., Ye, G., Shi, K., Wan, Y., Luo, C., Aihara, H., Geng, Q., Auerbach, A., & Li, F. (2020) · Nature · doi:10.1038/s41586-020-2179-y · Cited by 3,661
Research works printed but not linked: 1
- Negro, 2023, Knowledge Graph Applied
Cited by
Liu, Q., Feng, S., Huang, M., & Bhatti, U. A. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10983-0 · Cited by 19
Learning entity-query dependency and query classification for temporal knowledge graph prediction
He, M., Zhu, L., & Bai, L. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128016 · Cited by 6
Responsible models and indicators: challenges from artificial intelligence
Zhang, Y. (2023) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2023.1305692 · Cited by 2
Biomedical knowledge graphs driving new target identification and novel drug discovery
Dou, P., Yang, X., Jiang, Y., Xie, X., Geng, X., Huang, H., Li, H., & Li, S. (2026) · Targetome · doi:10.48130/targetome-0026-0025 · Cited by 1
Garimella, R. (2024) · Journal of High School Science · doi:10.64336/001c.120720
Metadata
- Authors
- Negro, A., Montagna, F., Teng, M. N., Neal, T., Thomas, S., King, S., & Khan, R.
- Affiliations
- University of South Florida (US)
- Corresponding author
- Negro, Alessandro
- Neal, Tempestt
- Venue
- Frontiers in Research Metrics and Analytics
- Publisher
- Frontiers Media SA
- Published
- 2023-08-03
- Volume
- 8
- Pages
- 1204801-1204801
- ISSN
- 2504-0537 (ISSN-L)
- PMC
- PMC10435856
- OpenAlex
- W4385519666
- Semantic Scholar
- d57ea3156a6a98048ad43b35d404f4fe193ec1ba
- Cited by
- 6
- References
- 67
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 2 · 2023: 1
- FWCI
- 0.22
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Research Metrics and Analytics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Science Foundation (2028051)
- Topics
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- COVID-19 pandemic, temporal knowledge graphs, knowledge graph evolution, ontology (information science), pandemic preparedness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs. https://app.cochat.ai/papers/10.3389/frma.2023.1204801
- The paper
- Negro, A., Montagna, F., Teng, M. N., Neal, T., Thomas, S., King, S., & Khan, R. (2023). Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs. Frontiers in Research Metrics and Analytics. https://doi.org/10.3389/frma.2023.1204801
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.