The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Punish/Reward: Learning with a Critic in Adaptive Threshold Systems
Abstract
An adaptive threshold element is able to "learn" a strategy of play for the game blackjack (twenty-one) with a performance close to that of the Thorp optimal strategy although the adaptive system has no prior knowledge of the game and of the objective of play. After each winning game the decisions of the adaptive system are "rewarded." After each losing game the decisions are "punished." Reward is accomplished by adapting while accepting the actual decision as the desired response. Punishment is accomplished by adapting while taking the desired response to be the opposite of that of the actual decision. This learning scheme is unlike "learning with a teacher" and unlike "unsupervised learning." It involves "bootstrap adaptation" or "learning with a critic." The critic rewards decisions which are members of successful chains of decisions and punishes other decisions. A general analytical model for learning with a critic is formulated and analyzed. The model represents bootstrap learning per se. Although the hypotheses on which the model is based do not perfectly fit blackjack learning, it is applied heuristically to predict adaptation rates with good experimental success. New applications are being explored for bootstrap learning in adaptive controls and multilayered adaptive systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
WIDROW, B., & HOFF, M. E. (1960) · doi:10.21236/ad0241531 · Cited by 4,009
Self-citationWidrow, B., Mantey, P. E., Griffiths, L. J., & Goode, B. B. (1967) · Proceedings of the IEEE · doi:10.1109/proc.1967.6092 · Cited by 1,051
Self-citationISODATA, A NOVEL METHOD OF DATA ANALYSIS AND PATTERN CLASSIFICATION
Ball, G. H., & Hall, D. J. (1965) · Defense Technical Information Center (DTIC) · Cited by 919
Torrieri, D. J. (1979) · doi:10.21236/ada082999 · Cited by 651
The Perceptron: A Model for Brain Functioning. I
Block, H. D. (1962) · Reviews of Modern Physics · doi:10.1103/revmodphys.34.123 · Cited by 396
State of the art in pattern recognition
Nagy, G. (1968) · Proceedings of the IEEE · doi:10.1109/proc.1968.6414 · Cited by 293
Spragins, J. (1966) · IEEE Transactions on Information Theory · doi:10.1109/tit.1966.1053865 · Cited by 187
Wilder, J. (1962) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.1962.16.2.326 · Cited by 162
On the advantages of the LMS spectrum analyzer over nonadaptive implementations of the sliding-DFT
Beaufays, F., & Widrow, B. C. (1995) · IEEE Transactions on Circuits and Systems I Fundamental Theory and Applications · doi:10.1109/81.382476 · Cited by 136
Self-citationLearning to recognize patterns without a teacher
Fralick, S. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053952 · Cited by 134
The use of an adaptive threshold element to design a linear optimal pattern classifier
Koford, J. S., & Groner, G. F. (1966) · IEEE Transactions on Information Theory · doi:10.1109/tit.1966.1053856 · Cited by 119
On pattern classification algorithms introduction and survey
Ho, Y., & Agrawala, A. K. (1968) · Proceedings of the IEEE · doi:10.1109/proc.1968.6823 · Cited by 83
Self-corrective character recognition system
George Nagy, A., & Shelton, G. (1966) · IEEE Transactions on Information Theory · doi:10.1109/tit.1966.1053864 · Cited by 68
Effects of Adaptation Parameters on Convergence Time and Tolerance for Adaptive Threshold Elements
Mays, C. H. (1964) · IEEE Transactions on Electronic Computers · doi:10.1109/pgec.1964.263850 · Cited by 29
Cited by
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
30 years of adaptive neural networks: perceptron, Madaline, and backpropagation
Widrow, B., & Lehr, M. A. (1990) · Proceedings of the IEEE · doi:10.1109/5.58323 · Cited by 2,242
Self-citationReinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,558
Prokhorov, D. V., & Wunsch, D. C. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.623201 · Cited by 1,186
Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof
Al-Tamimi, A., Lewis, F. L., & Abu-Khalaf, M. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.926614 · Cited by 1,075
Metadata
- Authors
- Widrow, B., Gupta, N. K., & Maitra, S.
- Venue
- IEEE Transactions on Systems, Man, and Cybernetics
- Published
- 1973-09
- Volume
- SMC-3
- Issue
- 5
- Pages
- 455-465
- ISSN
- 0018-9472 (ISSN-L)
- 2168-2909
- OpenAlex
- W2084378698
- Cited by
- 280
- References
- 14
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.