The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Evaluation methods for topic models
Open accessAbstract
A natural evaluation metric for statistical topic models is the probability of held-out documents given a trained model. While exact computation of this probability is intractable, several estimators for this probability have been used in the topic modeling literature, including the harmonic mean method and empirical likelihood method. In this paper, we demonstrate experimentally that commonly-used methods are unlikely to accurately estimate the probability of held-out documents, and propose two alternative methods that are both accurate and efficient.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Griffiths, T. L., & Steyvers, M. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0307752101 · Cited by 6,049
Hierarchical Dirichlet Processes
Teh, Y. W., Jordan, M. I., Beal, M. J., & Blei, D. M. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000302 · Cited by 2,120
Marginal Likelihood from the Gibbs Output
Chib, S. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476635 · Cited by 1,905
Approximate Bayesian Inference with the Weighted Likelihood Bootstrap
Newton, M. A., & Raftery, A. E. (1994) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1994.tb01956.x · Cited by 1,487
The author-topic model for authors and documents
Rosen‐Zvi, M., Griffiths, T. L., Steyvers, M., & Smyth, P. (2004) · Uncertainty in Artificial Intelligence · doi:10.5555/1036843.1036902 · Cited by 1,280
Sequential Monte Carlo Samplers
Del Moral, P., Doucet, A., & Jasra, A. (2006) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2006.00553.x · Cited by 1,239
Neal, R. M. (2001) · Statistics and Computing · doi:10.1023/a:1008923215028 · Cited by 1,172
Topic modeling: beyond bag-of-words
Wallach, H. M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143967 · Cited by 1,058
Self-citationPachinko allocation: DAG-structured mixture models of topic correlations
Li, W., & McCallum, A. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143917 · Cited by 609
Griffiths, T. L., Mark Steyvers, B., Blei, D. M., & Tenenbaum, J. B. (2004) · Cited by 501
Pachinko allocation: dag-structured mixture models of topic correlations
McCallum, A., & Li, W. (2007) · Cited by 143
Structured Topic Models for Language
Wallach, H. (2008) · Cited by 142
Self-citationEvaluating probabilities under high-dimensional latent variable models
Murray, I., & Salakhutdinov, R. (2008) · ERA · Cited by 61
Self-citation
Cited by
(2009) · Cited by 1,900
stm : An R Package for Structural Topic Models
Roberts, M. E., Stewart, B. M., & Tingley, D. (2019) · Journal of Statistical Software · doi:10.18637/jss.v091.i02 · Cited by 1,714
LDAvis: A method for visualizing and interpreting topics
Sievert, C., & Shirley, K. (2014) · Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces · doi:10.3115/v1/w14-3110 · Cited by 1,366
topicmodels : An R Package for Fitting Topic Models
Grün, B., & Hornik, K. (2011) · Journal of Statistical Software · doi:10.18637/jss.v040.i13 · Cited by 1,072
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2023) · arXiv · doi:10.48550/arxiv.2305.03514 · Cited by 839
Metadata
- Authors
- Wallach, H. M., Murray, I., Salakhutdinov, R., & Mimno, D.
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 1105-1112
- OpenAlex
- W2144100511
- Cited by
- 863
- References
- 14
- FWCI
- 20.68
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.