The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Modeling word burstiness using the Dirichlet distribution
Abstract
Multinomial distributions are often used to model text documents. However, they do not capture well the phenomenon that words in a document tend to appear in bursts: if a word appears once, it is more likely to appear again. In this paper, we propose the Dirichlet compound multinomial model (DCM) as an alternative to the multinomial. The DCM model has one additional degree of freedom, which allows it to capture burstiness. We show experimentally that the DCM is substantially better than the multinomial at modeling text data, measured by perplexity. We also show using three standard document collections that the DCM leads to better classification than the multinomial model. DCM performance is comparable to that obtained with multiple heuristic changes to the multinomial model. 1.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Machine learning in automated text categorization
Sebastiani, F. (2002) · ACM Computing Surveys · doi:10.1145/505282.505283 · Cited by 7,926
A vector space model for automatic indexing
Salton, G., Wong, A., & Yang, C. S. (1975) · Communications of the ACM · doi:10.1145/361219.361220 · Cited by 7,521 · cited 1×
“As most researchers do, we represent an individual document as a vector of word counts (Salton et al., 1975).”
BackgroundA STATISTICAL INTERPRETATION OF TERM SPECIFICITY AND ITS APPLICATION IN RETRIEVAL
SPARCK JONES, K. (1972) · Journal of Documentation · doi:10.1108/eb026526 · Cited by 4,505
Probabilistic Latent Semantic Indexing
Hofmann, T. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130370 · Cited by 3,988
A comparison of event models for naive bayes text classification
McCallum, A., & Nigam, K. (1998) · Cited by 3,289
Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology
Chao, Y. R., & Zipf, G. K. (1950) · Language · doi:10.2307/409735 · Cited by 2,193
Naive (Bayes) at forty: The independence assumption in information retrieval
Lewis, D. D. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0026666 · Cited by 2,104
An information-theoretic perspective of tf–idf measures
Aizawa, A. (2003) · Information Processing & Management · doi:10.1016/s0306-4573(02)00021-3 · Cited by 1,419
Tackling the poor assumptions of naive bayes text classifiers
Rennie, J. D. M., Shih, L., Teevan, J. B., & Karger, D. R. (2003) · Cited by 977
Automated learning of decision rules for text categorization
Apté, C., Damerau, F., & Weiss, S. M. (1994) · ACM Transactions on Information Systems · doi:10.1145/183422.183423 · Cited by 859
Estimating a Dirichlet Distribution
Minka, T. P. (2000) · Cited by 778
Distribution of content words and phrases in text and language modelling
KATZ, S. M. (1996) · Natural Language Engineering · doi:10.1017/s1351324996001246 · Cited by 259
Parametric models of linguistic count data
Jansche, M. (2003) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1075096.1075133 · Cited by 27
Teevan, J. B., & Karger, D. R. (2003) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/860435.860441 · Cited by 19
Cited by
Topics over time: a non-Markov continuous-time model of topical trends
Wang, X., & McCallum, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150450 · Cited by 1,121
PyMC: a modern, and comprehensive probabilistic programming framework in Python
Abril-Pla, O., Andreani, V., Carroll, C., Dong, L., Fonnesbeck, C. J., Kochurov, M., Kumar, R., Lao, J., Luhmann, C. C., Martin, O. A., et al. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1516 · Cited by 870
Optimal Thresholding of Classifiers to Maximize F1 Measure
Lipton, Z. C., Elkan, C., & Naryanaswamy, B. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-662-44851-9_15 · Cited by 596
Self-citationA dirichlet multinomial mixture model-based approach for short text clustering
Yin, J., & Wang, J. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623715 · Cited by 561
A Bayesian framework for word segmentation: Exploring the effects of context
Goldwater, S., Griffiths, T. L., & Johnson, M. (2009) · Cognition · doi:10.1016/j.cognition.2009.03.008 · Cited by 431
Metadata
- Authors
- Madsen, R. E., Kauchak, D., & Elkan, C.
- Venue
- Proceedings of the 22nd international conference on Machine learning - ICML '05
- Publisher
- ACM Press
- Published
- 2005
- Pages
- 545-552
- OpenAlex
- W2135631383
- Cited by
- 280
- References
- 16
- FWCI
- 8.11
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.