The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Learning agents for uncertain environments (extended abstract)
Abstract
This talk proposes a very simple "baseline architecture " for a learning agent that can handle stochastic, partially observable environments. The architecture uses reinforcement learning together with a method for representing temporal processes as graphical models. I will discuss methods for learning the parameters and structure of such representations from sensory inputs, and for computing posterior probabilities. Some open problems remain before we can try out the complete agent; more arise when we consider scaling up. A second theme of the talk will be whether reinforcement learning can provide a good model of animal and human learning. To answer this question, we must do inverse reinforcement learning: given the observed behaviour, what reward signal, if any, is being optimized? This seems to be a very interesting problem for the COLT, UAI, and ML communities, and has been addressed in econometrics under the heading of structural estimation of Markov decision processes. 1 Learnin...
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Artificial intelligence: a modern approach
(1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 21,974
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Decisions with Multiple Objectives: Preferences and Value Tradeoffs.
Tsoukias, A., Keeney, R. L., & Raiffa, H. (1994) · The Journal of the Operational Research Society · doi:10.2307/2584151 · Cited by 8,370
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,515
Artificial Intelligence: A Modern Approach
Ligeza, A. (1995) · Neurocomputing · doi:10.1016/0925-2312(95)90020-9 · Cited by 1,857
A model for reasoning about persistence and causation
Dean, T., & Kanazawa, K. (1989) · Computational Intelligence · doi:10.1111/j.1467-8640.1989.tb00324.x · Cited by 999
Gait and the energetics of locomotion in horses
Hoyt, D. F., & Taylor, C. R. (1981) · Nature · doi:10.1038/292239a0 · Cited by 908
Optimal control of Markov processes with incomplete state information
Åström, K. J. (1965) · Journal of Mathematical Analysis and Applications · doi:10.1016/0022-247x(65)90154-x · Cited by 854 · cited 1×
“Astrom (1965) proved that optimal decisions in POMDPs depend on the belief state b at each point in time, i.e., the posterior probability distribution over all possible actual states, given all evidence to date.”
BackgroundReinforcement Learning with Hierarchies of Machines
Parr, R., & Russell, S. (1997) · Cited by 618
Self-citationEstimation of Dynamic Labor Demand Schedules under Rational Expectations
Sargent, T. J. (1978) · Journal of Political Economy · doi:10.1086/260726 · Cited by 604
Learning the Structure of Dynamic Probabilistic Networks
Friedman, N., Murphy, K., & Russell, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.7374 · Cited by 561
Self-citationTractable Inference for Complex Stochastic Processes
Boyen, X., & Koller, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.7362 · Cited by 471
Adaptive Probabilistic Networks with Hidden Variables
Binder, J., Koller, D., Russell, S., & Kanazawa, K. (1997) · Machine Learning · doi:10.1023/a:1007421730016 · Cited by 377
Self-citationProceedings of the Fourteenth International Joint Conference on Artificial Intelligence - IJCAI-95
Bundy, A., & Lombart, V. (1995) · Cited by 364
Bee foraging in uncertain environments using predictive hebbian learning
Montague, P. R., Dayan, P., Person, C., & Sejnowski, T. J. (1995) · Nature · doi:10.1038/377725a0 · Cited by 311
A Mechanical Trigger for the Trot-Gallop Transition in Horses
Farley, C. T., & Taylor, C. R. (1991) · Science · doi:10.1126/science.1857965 · Cited by 297
Stochastic Simulation Algorithms for Dynamic Probabilistic Networks
Kanazawa, K., Koller, D., & Russell, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4965 · Cited by 219
Self-citationOvercoming Incomplete Perception with Utile Distinction Memory
McCallum, R. A. (1993) · Elsevier eBooks · doi:10.1016/b978-1-55860-307-3.50031-9 · Cited by 156
Approximating optimal policies for partially observable stochastic domains
Parr, R., & Russell, S. (1995) · Cited by 145
Self-citationOperant Conditioning in Skinnerbots
Touretzky, D. S., & Saksida, L. M. (1997) · Adaptive Behavior · doi:10.1177/105971239700500302 · Cited by 97
A Novel Reinforcement Model of Birdsong Vocalization Learning
Doya, K., & B, T. J. S. A. (1995) · Cited by 88
Escape, Avoidance, and Imitation: A Neural Network Approach
Schmajuk, N. A., & Zanutto, B. S. (1997) · Adaptive Behavior · doi:10.1177/105971239700600103 · Cited by 34
Space-efficient inference in dynamic probabilistic networks
Binder, J., Murphy, K. J., & Russell, S. (1997) · Cited by 33
Self-citation
Cited by
A survey of robot learning from demonstration
Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.10.024 · Cited by 3,227
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
Challenges of real-world reinforcement learning: definitions, benchmarks and analysis
Dulac-Arnold, G., Levine, N., Mankowitz, D. J., Li, J., Paduraru, C., Gowal, S., & Hester, T. (2021) · Machine Learning · doi:10.1007/s10994-021-05961-4 · Cited by 645
Research Priorities for Robust and Beneficial Artificial Intelligence
Russell, S., Dewey, D., & Tegmark, M. (2015) · AI Magazine · doi:10.1609/aimag.v36i4.2577 · Cited by 637
Self-citationA survey of inverse reinforcement learning: Challenges, methods and progress
Arora, S., & Doshi, P. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103500 · Cited by 610
Metadata
- Authors
- Russell, S.
- Venue
- Proceedings of the eleventh annual conference on Computational learning theory
- Publisher
- ACM
- Published
- 1998-07-24
- Pages
- 101-103
- OpenAlex
- W2031571562
- Cited by
- 297
- References
- 24
- FWCI
- 0.32
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.