The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Full-Text or Abstract? Examining Topic Coherence Scores Using Latent Dirichlet Allocation
Open accessAbstract
This paper assesses topic coherence and human topic ranking of uncovered latent topics from scientific publications when utilizing the topic model latent Dirichlet allocation (LDA) on abstract and full-text data. The coherence of a topic, used as a proxy for topic quality, is based on the distributional hypothesis that states that words with similar meaning tend to co-occur within a similar context. Although LDA has gained much attention from machine-learning researchers, most notably with its adaptations and extensions, little is known about the effects of different types of textual data on generated topics. Our research is the first to explore these practical effects and shows that document frequency, document word length, and vocabulary size have mixed practical effects on topic coherence and human topic ranking of LDA topics. We furthermore show that large document collections are less affected by incorrect or noise terms being part of the topic-word distributions, causing topics to be more coherent and ranked higher. Differences between abstract and full-text data are more apparent within small document collections, with differences as large as 90% high-quality topics for full-text data, compared to 50% high-quality topics for abstract data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Griffiths, T. L., & Steyvers, M. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0307752101 · Cited by 6,049
Blei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516
Harris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
Probabilistic Latent Semantic Indexing
Hofmann, T. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130370 · Cited by 3,988
Probabilistic latent semantic indexing
Hofmann, T. (1999) · Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/312624.312649 · Cited by 3,976
Software Framework for Topic Modelling with Large Corpora
Rek, R. R. U., & Sojka, P. (2010) · doi:10.13140/2.1.2393.1847 · Cited by 3,647
Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts
Grimmer, J., & Stewart, B. M. (2013) · Political Analysis · doi:10.1093/pan/mps028 · Cited by 3,297 · cited 1×
“Topic modeling approaches have proved to be very helpful in elucidating the key ideas within a set of documents [8]–[10], and they do so with greater speed and a quantitative rigor that would otherwise be possible only through a traditional narrative review [9].”
BackgroundExploring the Space of Topic Coherence Measures
Röder, M., Both, A., & Hinneburg, A. (2015) · Proceedings of the Eighth ACM International Conference on Web Search and Data Mining · doi:10.1145/2684822.2685324 · Cited by 2,224
(2009) · Cited by 1,900
Variational inference for Dirichlet process mixtures
Blei, D. M., & Jordan, M. I. (2006) · Bayesian Analysis · doi:10.1214/06-ba104 · Cited by 1,438
LDAvis: A method for visualizing and interpreting topics
Sievert, C., & Shirley, K. (2014) · Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces · doi:10.3115/v1/w14-3110 · Cited by 1,366
Online Learning for Latent Dirichlet Allocation
Hoffman, M. D., Blei, D. M., & Bach, F. (2010) · Cited by 1,363
Topics over time: a non-Markov continuous-time model of topical trends
Wang, X., & McCallum, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150450 · Cited by 1,121
Larsen, P. O., & von Ins, M. (2010) · Scientometrics · doi:10.1007/s11192-010-0202-z · Cited by 964
Evaluation methods for topic models
Wallach, H. M., Murray, I., Salakhutdinov, R., & Mimno, D. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553515 · Cited by 863
Automatic evaluation of topic coherence
Newman, D. J., Lau, J. H., Grieser, K., & Baldwin, T. (2010) · Cited by 846
Normalized (pointwise) mutual information in collocation extraction
Bouma, G. (2009) · Cited by 845
Fast collapsed gibbs sampling for latent dirichlet allocation
Porteous, I., Newman, D., Ihler, A., Asuncion, A., Smyth, P., & Welling, M. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401960 · Cited by 560
Operations for Learning with Graphical Models
Buntine, W. L. (1994) · Journal of Artificial Intelligence Research · doi:10.1613/jair.62 · Cited by 530
A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation
Teh, Y. W., Newman, D., & Welling, M. (2007) · doi:10.21236/ada629956 · Cited by 490
Exploring Topic Coherence over Many Models and Many Topics
Stevens, K., Kegelmeyer, P., Andrzejewski, D., & Buttler, D. J. (2012) · Cited by 470
Mixed-membership models of scientific publications
Erosheva, E., Fienberg, S., & Lafferty, J. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0307760101 · Cited by 442
On Smoothing and Inference for Topic Models
Asuncion, A., Welling, M., Smyth, P., & Teh, Y. W. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2662 · Cited by 434
Text Mining: Classification, Clustering, and Applications
(2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420059458 · Cited by 385
An analysis of the coherence of descriptors in topic modeling
O’Callaghan, D., Greene, D., Carthy, J., & Cunningham, P. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.02.055 · Cited by 370
Online Variational Inference for the Hierarchical Dirichlet Process
Wang, C., Paisley, J. W., & Blei, D. M. (2011) · Cited by 328
Latent Semantic Analysis: five methodological recommendations
Evangelopoulos, N., Zhang, X., & Prybutok, V. R. (2012) · European Journal of Information Systems · doi:10.1057/ejis.2010.61 · Cited by 306
Discovering themes and trends in transportation research using topic modeling
Sun, L., & Yin, Y. (2017) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2017.01.013 · Cited by 284
Evaluating Topic Coherence Using Distributional Semantics
Αλέτρας, Ν., & Stevenson, M. (2013) · Cited by 276
Interpretation and trust: designing model-driven visualizations for text analysis
Chuang, J., Ramage, D., Manning, C., & Heer, J. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207738 · Cited by 244
Distributed Inference for Latent Dirichlet Allocation
Newman, D. J., Smyth, P., Welling, M., & Asuncion, A. (2007) · Cited by 215
Chapman & Hall/CRC Data Mining and Knowledge Discovery Series
(2010) · Chapman & Hall/CRC data mining and knowledge discovery series · doi:10.1201/chdaminodis · Cited by 201
Studying the history of ideas using topic models
Hall, D., Jurafsky, D., & Manning, C. D. (2008) · Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08 · doi:10.3115/1613715.1613763 · Cited by 191
Creation of a highly detailed, dynamic, global model and map of science
Boyack, K. W., & Klavans, R. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.22990 · Cited by 168
Text analysis tools for identification of emerging topics and research gaps in conservation science
Westgate, M. J., Barton, P. S., Pierson, J. C., & Lindenmayer, D. B. (2015) · Conservation Biology · doi:10.1111/cobi.12605 · Cited by 123
Trends in Fisheries Science from 2000 to 2009: A Bibliometric Study
Jarić, I., Cvijanović, G., Knežević‐Jarić, J., & Lenhardt, M. B. (2012) · Reviews in Fisheries Science · doi:10.1080/10641262.2012.659775 · Cited by 53
A Historical Analysis of the Field of OR/MS using Topic ModelsPreprint
Gatti, C. J., Brooks, J. D., & Nurre, S. G. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.05154 · Cited by 48
Rusch, T. W., Hofmarcher, P., Hatzinger, R., & Hornik, K. (2013) · The Annals of Applied Statistics · doi:10.1214/12-aoas618 · Cited by 24
Antipodean agricultural and resource economics at 60: Trends in topics, authorship and collaboration
Polyakov, M., Gibson, F., & Pannell, D. J. (2016) · Australian Journal of Agricultural and Resource Economics · doi:10.1111/1467-8489.12152 · Cited by 10
Cited by
The Diversity–Innovation Paradox in Science
Hofstra, B., Kulkarni, V. V., Munoz-Najar Galvez, S., He, B., Jurafsky, D., & McFarland, D. A. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1915378117 · Cited by 975
An, Y., Yu, J. H., & James, S. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00507-3 · Cited by 403
A multi-perspective analysis of retractions in life sciences
Bhatt, B. (2021) · Scientometrics · doi:10.1007/s11192-021-03907-0 · Cited by 53
Open tracing tools: Overview and critical comparison
Janes, A., Li, X., & Lenarduzzi, V. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111793 · Cited by 28
Chen, H., Zhang, J., Zhang, X., & Zeng, N. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108651 · Cited by 8
Metadata
- Authors
- Syed, S., & Spruit, M.
- Venue
- 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 165-174
- OpenAlex
- W2782303220
- Cited by
- 400
- References
- 41
- FWCI
- 5.72
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.