Stochastic collapsed variational Bayesian inference for latent Dirichlet allocation
Open accessThe DOI matches its Crossref record; 27 of the 30 references it cites were found in a registry or an index.
Abstract
There has been an explosion in the amount of digital text information available in recent years, leading to challenges of scale for traditional inference algorithms for topic models. Recent advances in stochastic variational inference algorithms for latent Dirichlet allocation (LDA) have made it feasible to learn topic models on very large-scale corpora, but these methods do not currently take full advantage of the collapsed representation of the model. We propose a stochastic algorithm for collapsed variational Bayesian inference for LDA, which is simpler and more efficient than the state of the art method. In experiments on large-scale text corpora, the algorithm was found to converge faster and often to a better solution than previous methods. Human-subject experiments also demonstrated that the method can learn coherent topics in seconds on small corpora, facilitating the use of topic models in interactive document analysis software.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As outlined in [19], the SCVB0 inference algorithm is essentially equivalent to variants of the online-EM [23] algorithm with a modified prior.”
From Salmon provides fast and bias-aware quantification of transcript expression (2017) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643
Griffiths, T. L., & Steyvers, M. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0307752101 · Cited by 6,049 · cited 2×
“collapsed space are more efficient at improving held-out log probability than their uncollapsed counterparts, both per iteration and in wall-clock time per iteration [12, 24, 3].”
BackgroundMethodA View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187
Hierarchical Dirichlet Processes
Teh, Y. W., Jordan, M. I., Beal, M. J., & Blei, D. M. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000302 · Cited by 2,120
(2009) · Cited by 1,900
Online Learning for Latent Dirichlet Allocation
Hoffman, M. D., Blei, D. M., & Bach, F. (2010) · Cited by 1,469
Stochastic variational inference
Hoffman, M. D., Blei, D. M., Wang, C., & Paisley, J. W. (2013) · Journal of Machine Learning Research · doi:10.5555/2567709.2502622 · Cited by 1,333
Julia: A Fast Dynamic Language for Technical Computing
Anderson, T. A., Liu, H., Kuper, L., Totoni, E., Vitek, J., & Shpeisman, T. (2012) · arXiv (Cornell University) · doi:10.4230/lipics.ecoop.2017.4 · Cited by 627
Fast collapsed gibbs sampling for latent dirichlet allocation
Porteous, I., Newman, D., Ihler, A., Asuncion, A., Smyth, P., & Welling, M. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401960 · Cited by 560
Self-citation
Research works printed but not linked: 2
- B. Carpenter . Integrating out multinomial parameters in latent Dirichlet allocation and naive bayes for collapsed Gibbs sampling. Technical report , LingPipe , 2010 . B. Carpenter. Integrating out multinomial parameters in latent Dirichlet allocation and naive bayes for collapsed Gibbs sampling. Technical report, LingPipe, 2010.
- T. Minka . Power EP. Technical report , Microsoft Research , Cambridge , UK , 2004 . T. Minka. Power EP. Technical report, Microsoft Research, Cambridge, UK, 2004.
Cited by
Salmon provides fast and bias-aware quantification of transcript expression
Patro, R., Duggal, G., Love, M. I., Irizarry, R. A., & Kingsford, C. (2017) · Nature Methods · doi:10.1038/nmeth.4197 · Cited by 12,700
Joint Modeling of User Check-in Behaviors for Real-time Point-of-Interest Recommendation
Yin, H., Cui, B., Zhou, X., Wang, W., Huang, Z., & Sadiq, S. (2017) · ACM Transactions on Information Systems · doi:10.1145/2873055 · Cited by 171
RNA Sequencing Data: Hitchhiker's Guide to Expression Analysis
den Berge, K. V., Hembach, K., Soneson, C., Tiberi, S., Clement, L., Love, M. I., Patro, R., & Robinson, M. D. (2019) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-072018-021255 · Cited by 180
Patro, R., Duggal, G., & Kingsford, C. (2012) · KiltHub Repository · doi:10.1184/r1/6702233 · Cited by 100
Topic modelling for theme park online reviews: analysis of Disneyland
Luo, J. M., Vu, H. Q., Li, G., & Law, R. (2020) · Journal of Travel & Tourism Marketing · doi:10.1080/10548408.2020.1740138 · Cited by 95
Metadata
- Authors
- Foulds, J., Boyles, L., DuBois, C., Smyth, P., & Welling, M.
- Affiliations
- University of California, Irvine (US)
- University of Amsterdam (NL)
- Venue
- Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
- Publisher
- ACM
- Published
- 2013-08-11
- Pages
- 446-454
- OpenAlex
- W2001975024
- Cited by
- 126
- References
- 30
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 2 · 2023: 8 · 2022: 8 · 2021: 11 · 2020: 8 · 2019: 14 · 2018: 10 · 2017: 13 · 2016: 15 · 2015: 16 · 2014: 14 · 2013: 2
- FWCI
- 10.12
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Methods and Mixture Models
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- latent Dirichlet allocation, topic modeling, stochastic variational inference, topic coherence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Stochastic collapsed variational Bayesian inference for latent Dirichlet allocation. https://app.cochat.ai/papers/10.1145/2487575.2487697
- The paper
- Foulds, J., Boyles, L., DuBois, C., Smyth, P., & Welling, M. (2013). Stochastic collapsed variational Bayesian inference for latent Dirichlet allocation. Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. https://doi.org/10.1145/2487575.2487697
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.