An adaptive optimal controller for discrete-time Markov environments
Open accessIntegrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These milestones include the Backpropagation algorithm, the CMAC NN model, the Expectation Maximization algorithm, Temporal Difference learning, and the ID3 algorithm.
“These include the Backpropagation (BP) algorithm, the Cerebellar Model Articulation Controller (CMAC) NN model [11], the Expectation Maximization (EM) algorithm [115], the to-be-referred-to as Temporal Difference (TD) learning [478], and the Iterative Dichotomiser 3 (ID3) algorithm [373].”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [478]
Witten developed an RL approach to solve MDPs, inspired by animal behavior and learning theories, later referred to as Temporal Difference in Sutton's work.
“At the same time, Witten developed an RL approach to solve MDPs, inspired by animal behavior and learning theories [478], that was later referred to as Temporal Difference (TD) in Sutton's work [433,434].”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [478]
Cites
Nash, J. (1951) · The Annals of Mathematics · doi:10.2307/1969529 · Cited by 7,002 · see citing works
Nash, J. F. (1989) · Cambridge University Press eBooks · doi:10.1017/cbo9780511528231.007 · Cited by 5,661 · see citing works
Perceptrons: An Introduction to Computational Geometry
Minsky, M., & Papert, S. A. (1969) · Cited by 3,273 · see citing works
Punish/Reward: Learning with a Critic in Adaptive Threshold Systems
Widrow, B. C., Gupta, N. K., & Maitra, S. (1973) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1973.4309272 · Cited by 280 · see citing works
Use of Stochastic Automata for Parameter Self-Optimization with Multimodal Performance Criteria
Shapiro, I., & Narendra, K. S. (1969) · IEEE Transactions on Systems Science and Cybernetics · doi:10.1109/tssc.1969.300228 · Cited by 102 · see citing works
The two-armed-bandit problem with time-invariant finite memory
Cover, T. M., & Hellman, M. E. (1970) · IEEE Transactions on Information Theory · doi:10.1109/tit.1970.1054427 · Cited by 90 · see citing works
Human operators and automatic adaptive controllers: A comparative study on a particular control task
Witten, I. H., & Corbin, M. J. (1973) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(73)80010-0 · Cited by 12 · see citing works
Self-citationFinite-Time Performance of Some Two-Armed Bandit Controllers
Witten, I. H. (1973) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1973.5408504 · Cited by 10 · see citing works
Self-citationOn the Asymptotic Performances of Finite-State Two-Armed Bandit Controllers
(1974) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1974.4309343 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · see citing works
Boutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018) · Journal of Internet Services and Applications · doi:10.1186/s13174-018-0087-2 · Cited by 1,036 · see citing works
Neuronal Prediction of Opponent’s Behavior during Cooperative Social Interchange in Primates
Haroush, K., & Williams, Z. M. (2015) · Cell · doi:10.1016/j.cell.2015.01.045 · Cited by 210 · see citing works
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6 · see citing works
- Authors
- Witten, I. H.
- Affiliations
- University of Essex (GB)
- Corresponding author
- Witten, Ian H.
- ORCID
- Witten, Ian H.
- Venue
- Information and Control
- Publisher
- Elsevier BV
- Published
- 1977-08
- Volume
- 34
- Issue
- 4
- Pages
- 286-295
- ISSN
- 0019-9958 (ISSN-L)
- OpenAlex
- W2054940200
- Cited by
- 180
- References
- 9
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 8 · 2023: 10 · 2022: 11 · 2021: 11 · 2020: 12 · 2019: 9 · 2018: 11 · 2017: 3 · 2016: 1 · 2015: 5 · 2014: 3 · 2013: 3 · 2012: 6
- FWCI
- 1.44
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Information and Control (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Adaptive Dynamic Programming Control
- Keywords
- adaptive optimal control, Markov decision process, optimal control, expected reward maximization, reinforcement learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
