On the algorithmic implementation of stochastic discrimination
The DOI matches its Crossref record.
Abstract
Stochastic discrimination is a general methodology for constructing classifiers appropriate for pattern recognition. It is based on combining arbitrary numbers of very weak components, which are usually generated by some pseudorandom process, and it has the property that the very complex and accurate classifiers produced in this way retain the ability, characteristic of their weak component pieces, to generalize to new data. In fact, it is often observed, in practice, that classifier performance on test sets continues to rise as more weak components are added, even after performance on training sets seems to have reached a maximum. This is predicted by the underlying theory, for even though the formal error rate on the training set may have reached a minimum, more sophisticated measures intrinsic to this method indicate that classifier performance on both training and test sets continues to improve as complexity increases. We begin with a review of the method of stochastic discrimination as applied to pattern recognition. Through a progression of examples keyed to various theoretical issues, we discuss considerations involved with its algorithmic implementation. We then take such an algorithmic implementation and compare its performance, on a large set of standardized pattern recognition problems from the University of California Irvine, and Statlog collections, to many other techniques reported on in the literature, including boosting and bagging. In doing these studies, we compare our results to those reported in the literature by the various authors for the other methods, using the same data and study paradigms used by them. Included in the paper is an outline of the underlying mathematical theory of stochastic discrimination and a remark concerning boosting, which provides a theoretical justification for properties of that method observed in practice, including its ability to generalize.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Predictive learning via rule ensembles
Friedman, J. H., & Popescu, B. E. (2008) · The Annals of Applied Statistics · doi:10.1214/07-aoas148 · Cited by 1,315
“Details are presented in Ho and Kleinberg (1996) and Kleinberg (2000).”
BackgroundEnsemble Methods in Data Mining: Improving Accuracy Through Combining Predictions
Seni, G., & Elder, J. F. (2010) · Synthesis lectures on data mining and knowledge discovery · doi:10.2200/s00240ed1v01y200912dmk002 · Cited by 492
Managing computational complexity using surrogate models: a critical review
Alizadeh, R., Allen, J. K., & Mistree, F. (2020) · Research in Engineering Design · doi:10.1007/s00163-020-00336-7 · Cited by 453
Lee, S., Kim, J.-C., Jung, H.-S., Lee, M. J., & Lee, S. (2017) · Geomatics, Natural Hazards and Risk · doi:10.1080/19475705.2017.1308971 · Cited by 395
Ensembles of Learning Machines
Valentini, G., & Masulli, F. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-45808-5_1 · Cited by 276
Intelligence plays dice: Stochasticity is essential for machine learning
Sabuncu, M. R. (2020) · arXiv · doi:10.48550/arxiv.2008.07496 · Cited by 3
“llection of random models. One of the earliest proponents of this idea was Eugene Kleinberg, who formally studied a form of stochastic modeling, which gave rise to the popular random forest algorithm [34, 8]. In this framework, each model is restricted to a random subspace of the input domain. For example, a random su…”
Background
Metadata
- Authors
- Kleinberg, E. M.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2000-05
- Volume
- 22
- Issue
- 5
- Pages
- 473-490
- ISSN
- 0162-8828 (ISSN-L)
- OpenAlex
- W2077200366
- Cited by
- 131
- References
- 0
- FWCI
- 4.58
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.