Quantifying and filtering knowledge generated by literature based discovery
Open accessAbstract
BACKGROUND: Literature based discovery (LBD) automatically infers missed connections between concepts in literature. It is often assumed that LBD generates more information than can be reasonably examined. METHODS: We present a detailed analysis of the quantity of hidden knowledge produced by an LBD system and the effect of various filtering approaches upon this. The investigation of filtering combined with single or multi-step linking term chains is carried out on all articles in PubMed. RESULTS: The evaluation is carried out using both replication of existing discoveries, which provides justification for multi-step linking chain knowledge in specific cases, and using timeslicing, which gives a large scale measure of performance. CONCLUSIONS: While the quantity of hidden knowledge generated by LBD can be vast, we demonstrate that (a) intelligent filtering can greatly reduce the number of hidden knowledge pairs generated, (b) for a specific term, the number of single step connections can be manageable, and
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Researchers face a significant burden of manually interpreting potentially thousands of LBD associations, many of which are irrelevant or spurious.
“Consequently, researchers still face a significant burden of manually interpreting potentially thousands of associations, many of which are irrelevant or spurious [26].”
From SKiM-GPT: combining biomedical literature-based discovery with large language model hypothesis evaluation (2025) · cites this paper as [26]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Godsil, C., & Royle, G. (2001) · Graduate Texts in Mathematics · doi:10.1007/978-1-4613-0163-9 · Cited by 8,154 · see citing works
The unified medical language system (UMLS): integrating biomedical terminology
Bodenreider, O. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh061 · Cited by 4,356 · cited 2× · see citing works
“System Metathesaurus (UMLS) [9] is a large thesaurus which lists millions of biomedical and health related concepts using Concept Unique Identifiers (CUIs).”
BackgroundMethodAn overview of MetaMap: historical perspective and recent advances
Aronson, A. R., & Lang, F.-M. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.002733 · Cited by 1,519 · cited 2× · see citing works
“To identify the correct CUIs, they use MetaMap [11], a publicly available tool which assigns UMLS CUIs to terms, as well as mapping words to multi-word units where appropriate.”
MethodThe Oxford Guide to Practical Lexicography
Atkins, B. T. S., & Rundell, M. (2008) · doi:10.1093/oso/9780199277704.001.0001 · Cited by 949 · see citing works
Fish Oil, Raynaud's Syndrome, and Undiscovered Public Knowledge
Swanson, D. R. (1986) · Perspectives in biology and medicine · doi:10.1353/pbm.1986.0087 · Cited by 875 · cited 4× · see citing works
“Much of earlier LBD work restricted the knowledge base to a reduced time segment – for example, Gordon and Lindsay [5] restricted publications to the years 1983–1985 when they sought to replicate Swanson’s [2] fish oil – Raynaud disease connection.”
BackgroundInfluentialRindflesch, T. C., & Fiszman, M. (2003) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2003.11.003 · Cited by 519 · cited 1× · see citing works
“We base our relations on the output of the SemRep system [17] which uses underspecified syntactic processing and UMLS [9] domain knowledge to extract subject-relation-object triples (such as X-treats-Y or X-affects-Y ) from biomedical texts.”
MethodText mining: Generating hypotheses from MEDLINE
Srinivasan, P. (2003) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.10389 · Cited by 279 · see citing works
Weeber, M., Klein, H., den Berg, L. T. W. D. J., & Vos, R. (2001) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.1104 · Cited by 265 · cited 4× · see citing works
“Weeber et al. [10] filter out non-content words by switching from terms to UMLS CUIs.”
BackgroundMethodInfluentialA framework for understanding Latent Semantic Indexing (LSI) performance
Kontostathis, A., & Pottenger, W. M. (2005) · Information Processing & Management · doi:10.1016/j.ipm.2004.11.007 · Cited by 194 · cited 1× · see citing works
“For example, Kontostathi and Pottenger [14] investigate paths of linking terms generated by co-occurrence and Wilkowski et al. [15] show the feasibility of the approach on a single A and C pair.”
BackgroundGordon, M. D., & Lindsay, R. (1996) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199602)47:2 · Cited by 173 · see citing works
Digiacomo, R., Kremer, J. M., & Shah, D. M. (1989) · The American Journal of Medicine · doi:10.1016/0002-9343(89)90261-1 · Cited by 164 · cited 1× · see citing works
“Swanson [2] automatically discovered a previously unnoticed connection between fish oil and Raynaud disease, via a number of terms such as blood viscosity, platelet aggregation, vascular reactivity, a connection which was later verified [3]).”
BackgroundSomatomedin C and Arginine: Implicit Connections between Mutually Isolated Literatures
Swanson, D. R. (1990) · Perspectives in biology and medicine · doi:10.1353/pbm.1990.0031 · Cited by 112 · cited 1× · see citing works
“[Som-Arg] Somatomedin C – arginine [26]; 1960–1989, 27 direct connections.”
BackgroundLinking estrogen to Alzheimer's disease
Smalheiser, N. R., & Swanson, D. R. (1996) · Neurology · doi:10.1212/wnl.47.3.809 · Cited by 106 · cited 1× · see citing works
“[AD-est] Alzheimer’s disease – estrogen [29]; 1960–1995, 25 direct connections.”
Ranking indirect connections in literature‐based discovery: The role of medical subject headings
Swanson, D. R., Smalheiser, N. R., & Torvik, V. I. (2006) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20438 · Cited by 97 · see citing works
A new evaluation methodology for literature-based discovery systems
Yetisgen-Yildiz, M., & Pratt, W. (2008) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.12.001 · Cited by 93 · cited 2× · see citing works
“A cutoff date is chosen, hidden knowledge is generated from publications published prior to this date and the resulting pairs are compared to the new knowledge published after the cutoff (as identified by the used relation) [24].”
MethodOwen, J. M., Weeber, M., & Bruza, P. (2008) · Information science and knowledge management · doi:10.1007/978-3-540-68690-3 · Cited by 83 · see citing works
Context-driven automatic subgraph creation for literature-based discovery
Cameron, D., Kavuluru, R., Rindflesch, T. C., et al. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.01.014 · Cited by 79 · cited 2× · see citing works
“While not carrying out explicit synonym merging, Cameron et al. [12] manually add close terms to the source (A) and target (C) term in closed search (see bottom of Fig.”
BackgroundIndomethacin and Alzheimer's disease
Smalheiser, N. R., & Swanson, D. R. (1996) · Neurology · doi:10.1212/wnl.46.2.583 · Cited by 74 · cited 1× · see citing works
“[AD-INN] Alzheimer’s disease – indomethacin [28]; 1966–1996, 6 direct connections.”
Identifying plausible adverse drug reactions using knowledge extracted from the literature
Shang, N., Xu, H., Rindflesch, T. C., & Cohen, T. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.07.011 · Cited by 71 · cited 1× · see citing works
“Focusing on one type of discovery, adverse drug reactions, Shang et al. [13] employ only the INTERACTS_WITH and COMPARED_WITH relations within one step of their inference process.”
BackgroundDepression After Transient Ischemic Attack: A Clinically Distinct Subtype of Vascular Depression?
Rao, R. (1998) · Archives of General Psychiatry · doi:10.1001/archpsyc.55.8.752 · Cited by 68 · see citing works
Using Literature-based Discovery to Identify Novel Therapeutic Approaches
Hristovski, D., Rindflesch, T. C., & Peterlin, B. (2013) · Cardiovascular & Hematological Agents in Medicinal Chemistry · doi:10.2174/1871525711311010005 · Cited by 65 · cited 1× · see citing works
“Literature based discovery (LBD) automates the process of finding new connections (hidden connections) between existing knowledge, and thus can be used for disease candidate gene discovery, to find other uses of existing drugs, or for drug side effect prediction [4].”
BackgroundGraph-based methods for discovery browsing with semantic predications.
Wilkowski, B., Marcelo Fiszman, B., Miller, C. M., Hristovski, D., Arabandi, S., Rosemblat, G., & Rindflesch, T. C. (2011) · PubMed Central · Cited by 58 · see citing works
Fung, K. W., Hole, W. T., Nelson, S. J., et al. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1767 · Cited by 49 · cited 1× · see citing works
“The quality of synonyms in UMLS has been questioned [20], and we evaluate the synonym classes created (these are formed by gathering all synonyms, and their synonyms etc.”
BackgroundExploring relation types for literature-based discovery
Preiss, J., Stevenson, M., & Gaizauskas, R. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv002 · Cited by 42 · cited 2× · see citing works
“In turn, low precision accounts for the low F-measure; again, this is not unusual for large scale timeslicing results in LBD [6].”
BackgroundSelf-citationEvaluation of Literature-Based Discovery Systems
Yetisgen-Yildiz, M., & Pratt, W. (2008) · Information science and knowledge management · doi:10.1007/978-3-540-68690-3_7 · Cited by 27 · cited 2× · see citing works
“…to act as linking or target terms as these are thought to describe the type of desired hidden knowledge; for example Yetisgen-Yildiz and Pratt [8] allow both linking and target terms to be chemicals & drugs and genes & molecular sequences , but linking terms can also be disorders , physiology…”
BackgroundRicco, J. (1990) · Journal of Vascular Surgery · doi:10.1016/0741-5214(90)90229-4 · Cited by 27 · see citing works
Hu, X. T., Zhang, X., Yoo, I., et al. (2006) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972764.18 · Cited by 19 · cited 1× · see citing works
“[ RD-fsh ] Raynauddisease – fishoil [5, 10, 25]; 1960–1985, no direct connections.”
Seeking Informativeness in Literature Based Discovery
Preiss, J. (2014) · doi:10.3115/v1/w14-3417 · Cited by 15 · cited 1× · see citing works
“3 [22].”
Self-citation
Flagged first, then by citations
2 of the 30 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Smalheiser NR, Swanson DR. Assessing a gap in the biomedical literature: Magnesium deficiency and neurologic disease. Neurosci Res Commun. 1994; 15(1):1–9.
- Smalheiser NR, Swanson DR. Calcium-independent phospholipase A2 and schizophrenia. Arch Gen Psychiatr. 1997; 55(8):752–3.
Cited by
Sorted by citations
A Systematic Review on Literature-based Discovery
Thilakaratne, M., Falkner, K. E. K., & Atapattu, T. (2019) · ACM Computing Surveys · doi:10.1145/3365756 · Cited by 54 · see citing works
A systematic review on literature-based discovery workflow
Thilakaratne, M., Falkner, K. E. K., & Atapattu, T. (2019) · PeerJ Computer Science · doi:10.7717/peerj-cs.235 · Cited by 42 · see citing works
Jiang, Q., Zhang, F., Liu, X., et al. (2022) · Phytomedicine Plus · doi:10.1016/j.phyplu.2022.100296 · Cited by 5 · see citing works
Freeman, J., Millikin, R. J., Xu, L., Sharma, I., Moore, B., Lock, C., Shine George, K., Bal, A., Mohanty, C., & Stewart, R. (2025) · BMC Bioinformatics · doi:10.1186/s12859-025-06350-7 · Cited by 4 · see citing works
“Consequently, researchers still face a significant burden of manually interpreting potentially thousands of associations, many of which are irrelevant or spurious [26].”
Kim, S., Chen, J. Y., Cutello, V., et al. (2016) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2983323.2988540 · Cited by 1 · see citing works
“Preiss et al [3] describe a novel Literature-based discovery (LBD) method that extracts hidden connections between well-known knowledge, from published biological articles.”
BackgroundIs automatic detection of hidden knowledge an anomaly?
Preiss, J. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2815-4
“However, this is not an unexpected value: e.g. Preiss and Stevenson [17] obtain an F-measure between 1 × 10 − 03 and 3 × 10 − 03 for their large scale literature discovery.”
BackgroundSelf-citationMoore, B. M., Freeman, J., Millikin, R. J., Mohanty, C., George, K. W., Bal, A., Lock, C., Sauer, J., Spurgeon, M. E., Moore, D. L., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.05.730173
“…plausible-sounding but incorrect information (“hallucination”) 22 , 2) lack of transparency in the evidence used to arrive at their conclusions 23 , 3) non-peer-reviewed training data ( e.g. , social media misinformation), which compromises scientific provenance 23,24 , 4) inability to perform…”
- Authors
- Preiss, J., & Stevenson, M.
- Affiliations
- University of Sheffield (GB)
- Corresponding author
- Preiss, Judita
- ORCID
- Preiss, Judita
- Venue
- BMC Bioinformatics
- Published
- 2017-05
- Volume
- 18
- Issue
- S7
- Pages
- 249-249
- ISSN
- 1471-2105 (ISSN-L)
- PMC
- PMC5471938
- OpenAlex
- W2616598798
- Semantic Scholar
- e6718039b79eacefa582e5ca2f6b17b4e954d76c
- Cited by
- 11
- References
- 30
- Citations by year
- 2026: 1 · 2025: 1 · 2022: 1 · 2019: 7 · 2016: 1
- FWCI
- 0.78
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- BMC Bioinformatics (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- University of Salford Institutional Repository (University of Salford) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by)
- Funders
- Engineering and Physical Sciences Research Council (EP/J008427/1)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Keywords
- literature-based discovery, knowledge filtering, hidden knowledge, concept linking, biomedical literature
- MeSH terms
- Algorithms, Humans, Knowledge, Data Mining, Knowledge Discovery
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
