The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Punish/Reward: Learning with a Critic in Adaptive Threshold Systems
Abstract
An adaptive threshold element is able to "learn" a strategy of play for the game blackjack (twenty-one) with a performance close to that of the Thorp optimal strategy although the adaptive system has no prior knowledge of the game and of the objective of play. After each winning game the decisions of the adaptive system are "rewarded." After each losing game the decisions are "punished." Reward is accomplished by adapting while accepting the actual decision as the desired response. Punishment is accomplished by adapting while taking the desired response to be the opposite of that of the actual decision. This learning scheme is unlike "learning with a teacher" and unlike "unsupervised learning." It involves "bootstrap adaptation" or "learning with a critic." The critic rewards decisions which are members of successful chains of decisions and punishes other decisions. A general analytical model for learning with a critic is formulated and analyzed. The model represents bootstrap learning per se. Although the hypotheses on which the model is based do not perfectly fit blackjack learning, it is applied heuristically to predict adaptation rates with good experimental success. New applications are being explored for bootstrap learning in adaptive controls and multilayered adaptive systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
WIDROW, B., & HOFF, M. E. (1960) · doi:10.21236/ad0241531 · Cited by 4,009
Self-citationWidrow, B., Mantey, P. E., Griffiths, L. J., & Goode, B. B. (1967) · Proceedings of the IEEE · doi:10.1109/proc.1967.6092 · Cited by 1,051
Self-citationISODATA, A NOVEL METHOD OF DATA ANALYSIS AND PATTERN CLASSIFICATION
Ball, G. H., & Hall, D. J. (1965) · Defense Technical Information Center (DTIC) · Cited by 919
Torrieri, D. J. (1979) · doi:10.21236/ada082999 · Cited by 651
The Perceptron: A Model for Brain Functioning. I
Block, H. D. (1962) · Reviews of Modern Physics · doi:10.1103/revmodphys.34.123 · Cited by 396
State of the art in pattern recognition
Nagy, G. (1968) · Proceedings of the IEEE · doi:10.1109/proc.1968.6414 · Cited by 293
Spragins, J. (1966) · IEEE Transactions on Information Theory · doi:10.1109/tit.1966.1053865 · Cited by 187
Learning to recognize patterns without a teacher
Fralick, S. C. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053952 · Cited by 172
Wilder, J. (1962) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.1962.16.2.326 · Cited by 162
On the advantages of the LMS spectrum analyzer over nonadaptive implementations of the sliding-DFT
Beaufays, F., & Widrow, B. C. (1995) · IEEE Transactions on Circuits and Systems I Fundamental Theory and Applications · doi:10.1109/81.382476 · Cited by 136
Self-citation
Cited by
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
30 years of adaptive neural networks: perceptron, Madaline, and backpropagation
Widrow, B., & Lehr, M. A. (1990) · Proceedings of the IEEE · doi:10.1109/5.58323 · Cited by 2,242
Self-citationReinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,558
Prokhorov, D. V., & Wunsch, D. C. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.623201 · Cited by 1,186
Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof
Al-Tamimi, A., Lewis, F. L., & Abu-Khalaf, M. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.926614 · Cited by 1,075
Adaptive Dynamic Programming: An Introduction
Wang, F.-Y., Zhang, H., & Liu, D. (2009) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2009.932261 · Cited by 898
Handbook of Learning and Approximate Dynamic Programming
Si, J., Barto, A. G., Powell, W. B., & Wunsch, D. (2004) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544785 · Cited by 856
Neural networks for self-learning control systems
Nguyen, D. H., & Widrow, B. (1990) · IEEE Control Systems Magazine · doi:10.1109/37.55119 · Cited by 856
Self-citationZhang, H., Luo, Y., & Liu, D. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2027233 · Cited by 649
Bhasin, S., Kamalapurkar, R., Johnson, M., Vamvoudakis, K. G., Lewis, F. L., & Dixon, W. E. (2013) · Automatica · doi:10.1016/j.automatica.2012.09.019 · Cited by 642
Al-Tamimi, A., Lewis, F. L., & Abu-Khalaf, M. (2007) · Automatica · doi:10.1016/j.automatica.2006.09.019 · Cited by 611
Theory and applications of neural networks for industrial control systems
Fukuda, T., & Shibata, T. (1992) · IEEE Transactions on Industrial Electronics · doi:10.1109/41.170966 · Cited by 391
Wen, G., Chen, C. L. P., & Ge, S. S. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3002108 · Cited by 342
The Artificial Life Roots of Artificial Intelligence
Steels, L. (1993) · Artificial Life · doi:10.1162/artl.1993.1.1_2.75 · Cited by 330
A neuronal model of classical conditioning
Harry Klopf, A. (1988) · Psychobiology · doi:10.3758/bf03333113 · Cited by 318
Neural-net computing and the intelligent control of systems
PAO, Y.-H., PHILLIPS, S. M., & SOBAJIC, D. J. (1992) · International Journal of Control · doi:10.1080/00207179208934315 · Cited by 307
The hippocampal-striatal axis in learning, prediction and goal-directed behavior
Pennartz, C. M. A., Ito, R., Verschure, P. F. M. J., Battaglia, F. P., & Robbins, T. W. (2011) · Trends in Neurosciences · doi:10.1016/j.tins.2011.08.001 · Cited by 287
Adaptive-critic-based neural networks for aircraft optimal control
Balakrishnan, S. N., & Biega, V. (1996) · Journal of Guidance, Control, and Dynamics · doi:10.2514/3.21715 · Cited by 261
An adaptive optimal controller for discrete-time Markov environments
Witten, I. H. (1977) · Information and Control · doi:10.1016/s0019-9958(77)90354-0 · Cited by 180
Hierarchical control using networks trained with higher-level forward models
Wayne, G., & Abbott, L. F. (2014) · Neural Computation · doi:10.1162/neco_a_00639 · Cited by 14
Metadata
- Authors
- Widrow, B., Gupta, N. K., & Maitra, S.
- Venue
- IEEE Transactions on Systems, Man, and Cybernetics
- Published
- 1973-09
- Volume
- SMC-3
- Issue
- 5
- Pages
- 455-465
- ISSN
- 0018-9472 (ISSN-L)
- 2168-2909
- OpenAlex
- W2084378698
- Cited by
- 280
- References
- 14
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.