The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Clustering More than Two Million Biomedical Publications: Comparing the Accuracies of Nine Text-Based Similarity Approaches
Open accessAbstract
BACKGROUND: We investigate the accuracy of different similarity approaches for clustering over two million biomedical documents. Clustering large sets of text documents is important for a variety of information needs and applications such as collection management and navigation, summary and analysis. The few comparisons of clustering results from different similarity approaches have focused on small literature sets and have given conflicting results. Our study was designed to seek a robust answer to the question of which similarity approach would generate the most coherent clusters of a biomedical literature set of over two million documents. METHODOLOGY: We used a corpus of 2.15 million recent (2004-2008) records from MEDLINE, and generated nine different document-document similarity matrices from information extracted from their bibliographic records, including titles, abstracts and subject headings. The nine approaches were comprised of five different analytical techniques with two data sources. The five analytical techniques are cosine similarity using term frequency-inverse document frequency vectors (tf-idf cosine), latent semantic analysis (LSA), topic modeling, and two Poisson-based language models--BM25 and PMRA (PubMed Related Articles). The two data sources were a) MeSH subject headings, and b) words from titles and abstracts. Each similarity matrix was filtered to keep the top-n highest similarities per document and then clustered using a combination of graph layout and average-link clustering. Cluster results from the nine similarity approaches were compared using (1) within-cluster textual coherence based on the Jensen-Shannon divergence, and (2) two concentration measures based on grant-to-article linkages indexed in MEDLINE. CONCLUSIONS: PubMed's own related article approach (PMRA) generated the most coherent and most concentrated cluster solution of the nine text-based similarity approaches tested, followed closely by the BM25 approach using titles and abstracts. Approaches using only MeSH subject headings were not competitive with those based on titles and abstracts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Kohonen, T. (2001) · Springer Series in Information Sciences · doi:10.1007/978-3-642-56927-2 · Cited by 16,019
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,223
Term-weighting approaches in automatic text retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 9,740
Divergence measures based on the Shannon entropy
Lin, J. (1991) · IEEE Transactions on Information Theory · doi:10.1109/18.61115 · Cited by 5,140
Cumulated gain-based evaluation of IR techniques
Järvelin, K., & Kekäläinen, J. (2002) · ACM Transactions on Information Systems · doi:10.1145/582415.582418 · Cited by 4,841
Unsupervised Learning by Probabilistic Latent Semantic Analysis
Hofmann, T. (2001) · Machine Learning · doi:10.1023/a:1007617005950 · Cited by 2,456
Relevance weighting of search terms
Robertson, S. E., & Jones, K. S. (1976) · Journal of the American Society for Information Science · doi:10.1002/asi.4630270302 · Cited by 2,081
Using Linear Algebra for Intelligent Information Retrieval
Berry, M. W., Dumais, S. T., & O’Brien, G. W. (1995) · SIAM Review · doi:10.1137/1037127 · Cited by 1,481
Boyack, K. W., & Klavans, R. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21419 · Cited by 1,427
Self-citationA probabilistic model of information retrieval: development and comparative experiments
Sparck Jones, K., Walker, S., & Robertson, S. E. (2000) · Information Processing & Management · doi:10.1016/s0306-4573(00)00015-7 · Cited by 942
Mapping the backbone of science
Boyack, K. W., Klavans, R., & Börner, K. (2005) · Scientometrics · doi:10.1007/s11192-005-0255-6 · Cited by 937
Self-citationReexamining the cluster hypothesis: scatter/gather on retrieval results
Hearst, M. A., & Pedersen, J. O. (1996) · Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '96 · doi:10.1145/243199.243216 · Cited by 822
Textpresso: An Ontology-Based Information Retrieval and Extraction System for Biological Literature
Müller, H.-M., Kenny, E. E., & Sternberg, P. W. (2004) · PLoS Biology · doi:10.1371/journal.pbio.0020309 · Cited by 653
The use of hierarchic clustering in information retrieval
Jardine, N., & van Rijsbergen, C. J. (1971) · Information Processing & Management · doi:10.1016/0020-0271(71)90051-9 · Cited by 603
Index-driven similarity search in metric spaces (Survey Article)
Hjaltason, G. R., & Samet, H. (2003) · ACM Transactions on Database Systems · doi:10.1145/958942.958948 · Cited by 439
SOM_PAK: The Self-Organizing Map Program Package
Kohonen, T., Hynninen, J., Kangas, J., & Laaksonen, J. T. (1996) · Cited by 430
An Adaptation of the Vector-Space Model for Ontology-Based Information Retrieval
Castells, P., Fernandez, M., & Vallet, D. (2007) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.22 · Cited by 412
Text mining : applications and theory
Berry, M. W., & Kogan, J. (2010) · Cited by 287
OpenOrd: an open-source toolbox for large graph layout
Martin, S., Brown, W. M., Klavans, R., & Boyack, K. W. (2011) · SPIE Proceedings · doi:10.1117/12.871402 · Cited by 278
Self-citationA probabilistic model of information retrieval: development and comparative experiments
Sparck Jones, K., Walker, S., & Robertson, S. E. (2000) · Information Processing & Management · doi:10.1016/s0306-4573(00)00016-9 · Cited by 277
Combining the evidence of multiple query representations for information retrieval
Belkin, N. J., Kantor, P., Fox, E. A., & Shaw, J. A. (1995) · Information Processing & Management · doi:10.1016/0306-4573(94)00057-a · Cited by 268
PubMed related articles: a probabilistic topic-based model for content similarity
Lin, J., & Wilbur, W. J. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-423 · Cited by 243
On selecting a measure of retrieval effectiveness
Cooper, W. S. (1973) · Journal of the American Society for Information Science · doi:10.1002/asi.4630240204 · Cited by 224
Text Categorization Models for High-Quality Article Retrieval in Internal Medicine
Aphinyanaphongs, Y. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1641 · Cited by 177 · cited 1×
“This is as true in the world of biomedical literature as it is for any other literature; most studies related to enhancing the results of MEDLINE searches are very similar in nature to those being done in the broader information retrieval community [11,12,13].”
ResultIdentifying a better measure of relatedness for mapping science
Klavans, R., & Boyack, K. W. (2006) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20274 · Cited by 158
Self-citationFrom paragraph to graph: Latent semantic analysis for information visualization
Landauer, T. K., Laham, D., & Derr, M. A. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0400341101 · Cited by 147
Probabilistic topic decomposition of an eighteenth‐century American newspaper
Newman, D. J., & Block, S. (2006) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20342 · Cited by 127
Self-citationText similarity: an alternative way to search MEDLINE
Lewis, J. A., Ossowski, S., Hicks, J. M., Errami, M., & Garner, H. R. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl388 · Cited by 112
Ahlgren, P., & Colliander, C. (2009) · Journal of Informetrics · doi:10.1016/j.joi.2008.11.003 · Cited by 96
Visual conceptualizations and models of science
Börner, K., & Scharnhorst, A. (2009) · Journal of Informetrics · doi:10.1016/j.joi.2009.03.008 · Cited by 88
Self-citationMScanner: a classifier for retrieving Medline citations
Poulter, G., Rubin, D. L., Altman, R. B., & Seoighe, C. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-108 · Cited by 65
Ahlgren, P., & Jarneving, B. (2008) · Scientometrics · doi:10.1007/s11192-007-1935-1 · Cited by 56
Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries
Adam, N. R., Cassel, B., Yesha, Y., Furuta, R., & Weigle, M. C. (2016) · doi:10.1145/2910896 · Cited by 53
Biomedical knowledge navigation by literature clustering
Yamamoto, Y., & Takagi, T. (2006) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2006.07.004 · Cited by 47
A comparative study of citations and links in document classification
Couto, T., de Cristo, M. A. P., Gonçalves, M. A., Calado, P. P., Ziviani, N., de Moura, E. S., & Ribeiro-Neto, B. A. (2006) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/1141753.1141766 · Cited by 46
PuReD-MCL: a graph-based PubMed document clustering methodology
Θεοδοσίου, Θ., Darzentas, N., Angelis, L., & Ouzounis, C. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btn318 · Cited by 45
Semantically linking and browsing PubMed abstracts with gene ontology
Vanteru, B. C., Shaik, J. S., & Yeasin, M. (2008) · BMC Genomics · doi:10.1186/1471-2164-9-s1-s10 · Cited by 42
Combining Contents and Citations for Scientific Document Classification
Cao, M. D., & Gao, X. (2005) · Lecture notes in computer science · doi:10.1007/11589990_17 · Cited by 27
Harman, D. K., & Voorhees, E. M. (2006) · Annual Review of Information Science and Technology · doi:10.1002/aris.1440400111 · Cited by 24
Feature-based cluster validation for high-dimensional data
Kassab, R., & Lamirel, J.-C. (2008) · HAL · Cited by 20
Cited by
Citation-based clustering of publications using CitNetExplorer and VOSviewer
van Eck, N. J., & Waltman, L. (2017) · Scientometrics · doi:10.1007/s11192-017-2300-7 · Cited by 2,664
A new methodology for constructing a publication‐level classification system of science
Waltman, L., & van Eck, N. J. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22748 · Cited by 661
Systematic review automation technologies
Tsafnat, G., Glasziou, P., Choong, M. K., Dunn, A., Galgani, F., & Coiera, E. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-74 · Cited by 472
“Examples of document similarity measures include ones based on the terms that appear in the documents [66], and ones based”
BackgroundWang, Q., & Waltman, L. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.02.003 · Cited by 417
Klavans, R., & Boyack, K. W. (2017) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23734 · Cited by 334
Self-citation
Metadata
- Authors
- Boyack, K. W., Newman, D., Duhon, R. J., Klavans, R., Patek, M., Biberstine, J. R., Schijvenaars, B., Skupin, A., Ma, N., & Börner, K.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-03-17
- Volume
- 6
- Issue
- 3
- Pages
- e18029
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2025365931
- Cited by
- 271
- References
- 42
- FWCI
- 13.51
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.