The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Large-Scale Bayesian Logistic Regression for Text Categorization
Abstract
Logistic regression analysis of high-dimensional data, such as natural language text, poses computational and statistical challenges. Maximum likelihood estimation often fails in these applications. We present a simple Bayesian logistic regression approach that uses a Laplace prior to avoid overfitting and produces sparse predictive models for text data. We apply this approach to a range of document classification problems and show that it produces compact predictive models at least as effective as those produced by support vector machine classifiers or ridge logistic regression combined with feature selection. We describe our model fitting algorithm, our open source implementations (BBR and BMR), and experimental results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Neural networks for pattern recognition
(1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,538
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
Term-weighting approaches in automatic text retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 9,740
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
An algorithm for suffix stripping
Porter, M. F. (1980) · Program · doi:10.1108/eb046814 · Cited by 8,096
Text categorization with Support Vector Machines: Learning with many relevant features
Joachims, T. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0026683 · Cited by 7,999
Machine learning in automated text categorization
Sebastiani, F. (2002) · ACM Computing Surveys · doi:10.1145/505282.505283 · Cited by 7,926
Linear and nonlinear programming.
(1986) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(86)90092-3 · Cited by 4,805
A Comparative Study on Feature Selection in Text Categorization
Yang, Y., & Pedersen, J. (1997) · Cited by 4,790
Jurafsky, D. S., & Martin, J. (2000) · Cited by 4,132
Speech and language processing
(2010) · Day-to-Day Dyslexia in the Classroom · doi:10.4324/9780203461891_chapter_3 · Cited by 3,544
RCV1: A New Benchmark Collection for Text Categorization Research
Lewis, D., Yang, Y., Rose, T. G., & Li, F. (2004) · Goldsmiths (University of London) · doi:10.5555/1005332.1005345 · Cited by 2,503
Self-citationAn extensive empirical study of feature selection metrics for text classification
Forman, G. (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944974 · Cited by 2,394
BoosTexter: A Boosting-based System for Text Categorization
Schapire, R. E., & Singer, Y. (2000) · Machine Learning · doi:10.1023/a:1007649029923 · Cited by 2,188
Naive (Bayes) at forty: The independence assumption in information retrieval
Lewis, D. D. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0026666 · Cited by 2,104
Self-citationRidge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.2307/1267351 · Cited by 1,955
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/15324430152748236 · Cited by 1,819
Ridge Estimators in Logistic Regression
Cessie, S. L., & Houwelingen, J. C. V. (1992) · Applied Statistics · doi:10.2307/2347628 · Cited by 1,680
Bayesian Variable Selection in Linear Regression
Mitchell, T. J., & Beauchamp, J. J. (1988) · Journal of the American Statistical Association · doi:10.1080/01621459.1988.10478694 · Cited by 1,424
Learning to Classify Text Using Support Vector Machines
Joachims, T. (2002) · Springer eBooks · doi:10.1007/978-1-4615-0907-3 · Cited by 1,366
On Model Selection Consistency of Lasso
ZhaoPeng, & Yubin, Y. (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248637 · Cited by 996
Inference and disputed authorship : The Federalist
Mosteller, F. C., & Wallace, D. L. (1964) · Medical Entomology and Zoology · Cited by 904
Sparse multinomial logistic regression: fast algorithms and generalization bounds
Krishnapuram, B., Carin, L., Figueiredo, M. A. T., & Hartemink, A. J. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.127 · Cited by 877
L 1-Regularization Path Algorithm for Generalized Linear Models
Park, M. Y., & Hastie, T. (2007) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2007.00607.x · Cited by 838
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (2000) · Technometrics · doi:10.1080/00401706.2000.10485983 · Cited by 808
A comparison of algorithms for maximum entropy parameter estimation
Malouf, R. (2002) · proceeding of the 6th conference on Natural language learning - COLING-02 · doi:10.3115/1118853.1118871 · Cited by 655
Weiss, S. M., Indurkhya, N., Zhang, T., & Damerau, F. J. (2005) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · doi:10.1007/978-0-387-34555-0 · Cited by 583
Automatic Indexing: An Experimental Inquiry
Maron, M. E. (1961) · Journal of the ACM · doi:10.1145/321075.321084 · Cited by 570 · cited 2×
“Examples include naive Bayes (Maron 1961; Lewis 1998) and the Rocchio algorithm (Rocchio 1971).”
BackgroundAdaptive sparseness for supervised learning
Figueiredo, M. A. T. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1227989 · Cited by 536
Training algorithms for linear text classifiers
Lewis, D. D., Schapire, R. E., Callan, J. P., & Papka, R. (1996) · Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '96 · doi:10.1145/243199.243277 · Cited by 520
Self-citationText categorization with support vector machines
Joachims, T. (1997) · Technische Universität Dortmund Eldorado (Technische Universität Dortmund) · doi:10.17877/de290r-5097 · Cited by 507
An Equivalence Between Sparse Approximation and Support Vector Machines
Girosi, F. (1998) · Neural Computation · doi:10.1162/089976698300017269 · Cited by 506
Lowe, H. J. (1994) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1994.03510380059038 · Cited by 468
Survey of Text Mining: Clustering, Classification, and Retrieval
Berry, M. W., & Castellanos, M. (2007) · Cited by 406
A simple and efficient algorithm for gene selection using sparse logistic regression
Shevade, S. K., & Keerthi, S. S. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg308 · Cited by 401
Text Categorization Based on Regularized Linear Classification Methods
Zhang, T., & Oles, F. J. (2001) · Information Retrieval · doi:10.1023/a:1011441423217 · Cited by 379
Problems due to Small Samples and Sparse Data in Conditional Logistic Regression Analysis
Greenland, S., Schwartzbaum, J. A., & Finkle, W. D. (2000) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a010240 · Cited by 370
Evaluating and optimizing autonomous text classification systems
Lewis, D. D. (1995) · Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '95 · doi:10.1145/215206.215366 · Cited by 308
Self-citationThe role of data mining in pharmacovigilance
Hauben, M., Madigan, D., Gerrits, C. M., Walsh, L., & Van Puijenbroek, E. P. (2005) · Expert Opinion on Drug Safety · doi:10.1517/14740338.4.5.929 · Cited by 276
Self-citationThe Statistical Analysis of Discrete Data.
Iyer, R., Santner, T. J., & Duffy, D. E. (1991) · The Statistician · doi:10.2307/2348290 · Cited by 195
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303322753670 · Cited by 189
Detecting Differentially Expressed Genes in Microarrays Using Bayesian Model Selection
Ishwaran, H., & Rao, J. S. (2003) · Journal of the American Statistical Association · doi:10.1198/016214503000224 · Cited by 165
Discussion of "Least Angle Regression" by Efron, Hastie, Johnstone and Tibshirani
Turlach, B. A. (2004) · UWA Profiles and Research Repository (University of Western Australia) · Cited by 153
Relative Loss Bounds for Multidimensional Regression Problems
Kivinen, J., & Warmuth, M. K. (2001) · Machine Learning · doi:10.1023/a:1017938623079 · Cited by 135
Bayesian Classification of Tumours by Using Gene Expression Data
Mallick, B. K., Ghosh, D., & Ghosh, M. (2005) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2005.00498.x · Cited by 88
Gradient Directed Regularization for Linear Regression and Classi…cation
FriedmanBogdan, J. H., & Popescu, E. (2004) · Cited by 86
Robustness of regularized linear classification methods in text categorization
Zhang, J., & Yang, Y. (2003) · doi:10.1145/860435.860471 · Cited by 72
Bayesian Multinomial Logistic Regression for Author Identification
Madigan, D. (2005) · AIP conference proceedings · doi:10.1063/1.2149832 · Cited by 68
Self-citationFast Robust Logistic Regression for Large Sparse Datasets with Binary Outputs
Komarek, P., & Moore, A. (2003) · Cited by 65
Bayesian learning of sparse classifiers
Figueiredo, M. A. T., & Jain, A. K. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2001.990453 · Cited by 65
American journal of epidemiology
(1993) · Journal of the American Dietetic Association · doi:10.1016/0002-8223(93)91975-v · Cited by 52
Recovering genetic regulatory networks from micro-array data and location analysis data.
Li, F., & Yang, Y. (2004) · PubMed · Cited by 18
Cited by
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
“Some of the new classification algorithms in WEKA 3.6 include • Bayesian logistic regression [13]: the BLR method for text categorization, with both Gaussian and Laplace priors.”
InfluentialNearly unbiased variable selection under minimax concave penalty
Zhang, C.-H. (2010) · The Annals of Statistics · doi:10.1214/09-aos729 · Cited by 3,835
Regression Shrinkage and Selection via The Lasso: A Retrospective
Tibshirani, R. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2011.00771.x · Cited by 3,589
A weakly informative default prior distribution for logistic and other regression models
Gelman, A., Jakulin, A., Pittau, M. G., & Su, Y.-S. (2008) · The Annals of Applied Statistics · doi:10.1214/08-aoas191 · Cited by 1,790
The Development and Psychometric Properties of LIWC2015
Pennebaker, J. W., Boyd, R. L., Jordan, K., & Blackburn, K. (2015) · University of Texas at Austin · doi:10.15781/t25p41 · Cited by 1,674
Metadata
- Authors
- Genkin, A., Lewis, D. D., & Madigan, D.
- Venue
- Technometrics
- Publisher
- Informa UK Limited
- Published
- 2007-08
- Volume
- 49
- Issue
- 3
- Pages
- 291-304
- ISSN
- 0040-1706 (ISSN-L)
- 1537-2723
- OpenAlex
- W2166183437
- Cited by
- 808
- References
- 56
- FWCI
- 58.70
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.