The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Learning to perceive and act by trial and error
Open accessOverview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, sensations from the world are aliased (10, 11) and do not convey locations directly.”
From Space is a latent sequence: A theory of the hippocampus (2024) · cites this paper as [10]
Citations
Cites
Adaptation in Natural and Artificial Systems
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,340
The Ecological Approach to Visual Perception
Bornstein, M. H., & Gibson, J. J. (1980) · The Journal of Aesthetics and Art Criticism · doi:10.2307/429816 · Cited by 22,935
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
The Ecological Approach to Visual Perception: Classic Edition
Gibson, J. J. (2014) · doi:10.4324/9781315740218 · Cited by 12,132
A robust layered control system for a mobile robot
Brooks, R. (1986) · IEEE Journal on Robotics and Automation · doi:10.1109/jra.1986.1087032 · Cited by 7,799
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,674
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,325
Advances in Neural Information Processing Systems 37
Touretzky, D. S. (2024) · doi:10.52202/079017 · Cited by 2,874
The Ecological Approach To Visual Perception
Gibson, J. J. (2013) · Psychology Press eBooks · doi:10.4324/9780203767764 · Cited by 2,622
Research works printed but not linked: 5
- Anderson, C. W. (1989). Towers of hanoi with connectionist networks: Learning new features. Proceedings of the Sixth International Conference on Machine Learning (pp. 345?350). San Mateo, CA: Morgan Kaufmann.
- Bellman, R. E. (1957). Dynamic programming. Princeton, NJ: Princeton University Press.
- Nilsson, N. J. (1989). Action networks. Proceedings of the Rochester Planning Workshop (Technical Report 284) (pp. 36?68). University of Rochester, Department of Computer Science.
- Whitehead, S. D., & Ballard, D. H. (1991). A study of cooperative mechanisms for faster reinforcement learning (Technical Report TR 365). Rochester, NY: University of Rochester, Department of Computer Science.
- Williams, R. J. (1987). Reinforcement-learning connectionist systems (Technical Report NU-CCS-87?3). Boston, MA: Northeastern University, College of Computer Science.
Cited by
Sorted by citations
Self-improving reactive agents based on reinforcement learning, planning and teaching
Lin, L.-J. (1992) · Machine Learning · doi:10.1007/bf00992699 · Cited by 1,564
Deictic codes for the embodiment of cognition
Ballard, D. H., Hayhoe, M. M., Pook, P. K., & Rao, R. P. N. (1997) · Behavioral and Brain Sciences · doi:10.1017/s0140525x97001611 · Cited by 1,082
Self-citationReinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 841
Memory Representations in Natural Tasks
Ballard, D. H., Hayhoe, M. M., & Pelz, J. B. (1995) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1995.7.1.66 · Cited by 817
Self-citationDyna, an Integrated Architecture for Learning, Planning, and Reacting
Sutton, R. S. (1991) · ACM SIGART Bulletin · doi:10.1145/122344.122377 · Cited by 777
Acting Optimally in Partially Observable Stochastic Domains
Cassandra, A. R., Kaelbling, L. P., & Littman, M. L. (1994) · Cited by 570
Reinforcement learning is direct adaptive optimal control
Sutton, R. S., Barto, A. G., & Williams, R. J. (1992) · IEEE Control Systems · doi:10.1109/37.126844 · Cited by 537
Thrun, S., & Mitchell, T. M. (1995) · Robotics and Autonomous Systems · doi:10.1016/0921-8890(95)00004-y · Cited by 536
Dearden, R., Friedman, N., & Russell, S. (1998) · Cited by 363
Reinforcement learning with perceptual aliasing: the perceptual distinctions approach
Chrisman, L. (1992) · Cited by 316
Hand-eye coordination during sequential tasks
Ballard, D. H., Hayhoe, M., Li, F., & Whitehead, S. D. (1992) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.1992.0111 · Cited by 310
Self-citationRobot shaping: developing autonomous agents through learning
Dorigo, M., & Colombetti, M. (1994) · Artificial Intelligence · doi:10.1016/0004-3702(94)90047-7 · Cited by 238
The role of exploration in learning control
Thrun, S. (1992) · Cited by 226
Instance-Based Utile Distinctions for Reinforcement Learning with Hidden State
McCallum, R. A. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50055-4 · Cited by 182
Visual surveillance in a dynamic and uncertain world
Buxton, H., & Gong, S. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(95)00041-0 · Cited by 169
An optimal estimation approach to visual perception and learning
Rao, R. P. N. (1999) · Vision Research · doi:10.1016/s0042-6989(98)00279-x · Cited by 173
The implementation of visual routines
Roelfsema, P. R., Lamme, V. A. F., & Spekreijse, H. (2000) · Vision Research · doi:10.1016/s0042-6989(00)00004-3 · Cited by 162
Self-Improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching
Lin, L.-J. (1992) · Springer eBooks · doi:10.1007/978-1-4615-3618-5_5 · Cited by 176
Self-Improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching
Lin, L.-J. (1992) · Machine Learning · doi:10.1023/a:1022628806385 · Cited by 157
Learning and understanding dynamic scene activity: a review
Buxton, H. (2003) · Image and Vision Computing · doi:10.1016/s0262-8856(02)00127-0 · Cited by 157
Introduction: The challenge of reinforcement learning
Sutton, R. S. (1992) · Machine Learning · doi:10.1007/bf00992695 · Cited by 150
Reinforcement learning of non-Markov decision processes
Whitehead, S. D., & Lin, L.-J. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00012-p · Cited by 135
Self-citationActive object recognition by view integration and reinforcement learning
Paletta, L., & Pinz, A. J. (2000) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(99)00079-2 · Cited by 128
Comparing exploration strategies for Q-learning in random stochastic mazes
Tijsma, A. D., Drugan, M. M., & Wiering, M. (2016) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2016.7849366 · Cited by 128
Power and the limits of reactive agents
Nolfi, S. (2002) · Neurocomputing · doi:10.1016/s0925-2312(01)00598-7 · Cited by 120
Introduction: The Challenge of Reinforcement Learning
Sutton, R. S. (1992) · Springer eBooks · doi:10.1007/978-1-4615-3618-5_1 · Cited by 114
The Agent-Based Approach: A New Direction for Computational Models of Development
Schlesinger, M., & Parisi, D. (2001) · Developmental Review · doi:10.1006/drev.2000.0520 · Cited by 114
Reinforcement learning control
Barto, A. G. (1994) · Current Opinion in Neurobiology · doi:10.1016/0959-4388(94)90138-4 · Cited by 112
Thrun, S., & Mitchell, T. M. (1995) · Springer eBooks · doi:10.1007/978-3-642-79629-6_7 · Cited by 107
Short-term gains, long-term pains: How cues about state aid learning in dynamic environments
Gureckis, T. M., & Love, B. C. (2009) · Cognition · doi:10.1016/j.cognition.2009.03.013 · Cited by 98
Incremental learning of subtasks from unsegmented demonstration
Grollman, D. H., & Jenkins, O. C. (2010) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2010.5650500 · Cited by 88
Exploration of Multi-State Environments: Local Measures and Back-Propagation of Uncertainty
Meuleau, N., & Bourgine, P. (1999) · Machine Learning · doi:10.1023/a:1007541107674 · Cited by 91
Scaling Up Reinforcement Learning for Robot Control
Lin, L.-J. (1993) · Elsevier eBooks · doi:10.1016/b978-1-55860-307-3.50030-7 · Cited by 81
Learning in noise: Dynamic decision-making in a variable environment
Gureckis, T. M., & Love, B. C. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2009.02.004 · Cited by 88
Reinforcement learning for the adaptive control of perception and action
Whitehead, S. D. (1992) · UR Research (University of Rochester) · Cited by 77
Self-citationNatural communication with robots
Torrance, M. (1994) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 76
Discovery as Autonomous Learning from the Environment
Shen, W. (1993) · Machine Learning · doi:10.1007/bf00993064 · Cited by 75
Computational Foundations of Natural Intelligence
van Gerven, M. A. J. (2017) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2017.00112 · Cited by 76
Efficient reinforcement learning
Fiechter, C.-N. (1994) · Conference on Learning Theory · doi:10.1145/180139.181019 · Cited by 79
Time Limits in Reinforcement LearningPreprint
Pardo, F., Tavakoli, A., Levdik, V., & Kormushev, P. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.00378 · Cited by 99
Reinforcement Learning is Direct Adaptive Optimal Control
Sutton, R. S., Barto, A. G., & Williams, R. J. (1991) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.1991.4791776 · Cited by 69
An algebraic approach to abstraction in reinforcement learning
Ravindran, B., & Barto, A. G. (2004) · Cited by 65
Toward the Formal Foundation of Ant Programming
Birattari, M., Di, G. A., & Dorigo, M. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45724-0_16 · Cited by 63
Active learning in Gaussian process interpolation of potential energy surfaces
Uteva, E., Graham, R. S., Wilkinson, R. D., & Wheatley, R. J. (2018) · The Journal of Chemical Physics · doi:10.1063/1.5051772 · Cited by 64
Learning goal-directed sensory-based navigation of a mobile robot
Tani, J., & Fukumura, N. (1994) · Neural Networks · doi:10.1016/0893-6080(94)90112-0 · Cited by 62
Exploiting the Power of Sensory-Motor Coordination
Nolfi, S., & Parisi, D. (1999) · Lecture notes in computer science · doi:10.1007/3-540-48304-7_24 · Cited by 60
A framework for meta-level control in multi-agent systems
Raja, A., & Lesser, V. (2007) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-006-9008-z · Cited by 58
Baldassarre, G. (2002) · Cognitive Systems Research · doi:10.1016/s1389-0417(01)00039-0 · Cited by 57
Ognibene, D., & Baldassarre, G. (2014) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2014.2341351 · Cited by 55
Learning movement primitive attractor goals and sequential skills from kinesthetic demonstrations
Manschitz, S., Kober, J., Gienger, M., & Peters, J. (2015) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2015.07.005 · Cited by 56
Lashgari, A., Fouladgar, M. M., Yazdani–Chamzini, A., & Skibniewski, M. J. (2011) · Journal of Civil Engineering and Management · doi:10.3846/13923730.2011.628687 · Cited by 53
Inferring Finite Automata with Stochastic Output Functions and an Application to Map Learning
Dean, T., Angluin, D., Basye, K., Engelson, S. P., Kaelbling, L. P., Kokkevis, E., & Maron, O. (1995) · Machine Learning · doi:10.1023/a:1022874607797 · Cited by 57
Anticipations Control Behavior: Animal Behavior in an Anticipatory Learning Classifier System
Butz, M. V., & Hoffmann, J. (2002) · Adaptive Behavior · doi:10.1177/1059712302010002001 · Cited by 50
Action-Selection Method for Reinforcement Learning Based on Cuckoo Search Algorithm
Abed-alguni, B. H. (2017) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-017-2873-8 · Cited by 48
Histogram of Oriented Uniform Patterns for robust place recognition and categorization
Fazl-Ersi, E., & Tsotsos, J. K. (2012) · The International Journal of Robotics Research · doi:10.1177/0278364911434936 · Cited by 45
Toward a Model of Intelligence as an Economy of Agents
Baum, E. B. (1999) · Machine Learning · doi:10.1023/a:1007593124513 · Cited by 41
Anytime Learning and Adaptation of Structured Fuzzy Behaviors
Bonarini, A. (1997) · Adaptive Behavior · doi:10.1177/105971239700500304 · Cited by 40
Evolving Robots Able To Integrate Sensory-Motor Information Over Time
Nolfi, S. (2001) · Theory in Biosciences · doi:10.1078/1431-7613-00046 · Cited by 39
Closed-Loop Learning of Visual Control Policies
Jodogne, S., & Piater, J. H. (2006) · Cited by 39
The anticipatory classifier system and genetic generalization
Butz, A. M. V., Goldberg, B. D. E., & Stolzmann, C. W. (2002) · Natural Computing · doi:10.1023/a:1021330114221 · Cited by 39
Modular inverse reinforcement learning for visuomotor behavior
Rothkopf, C. A., & Ballard, D. H. (2013) · Biological Cybernetics · doi:10.1007/s00422-013-0562-6 · Cited by 38
Self-citationGaskett, C. (2002) · PubMed Central · doi:10.25911/5d7a2a09a7dfd · Cited by 38
Multi-agent Reinforcement Learning for Simulating Pedestrian Navigation
Martínez-Gil, F., Lozano, M., & Fernández, F. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-28499-1_4 · Cited by 36
Strategies for simulating pedestrian navigation with multiple reinforcement learning agents
Martínez-Gil, F., Lozano, M., & Fernández, F. (2014) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-014-9252-6 · Cited by 35
Siddique, A., Browne, W. N., & Grimshaw, G. M. (2021) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2021.3102241 · Cited by 35
Rao, R. P. N., & Fuentes, O. (1998) · Machine Learning · doi:10.1023/a:1007492624519 · Cited by 33
Active control in interrupted dynamic spatial orientation: The detection of orientation change
Larish, J. F., & Andersen, G. J. (1995) · Perception & Psychophysics · doi:10.3758/bf03213078 · Cited by 32
Learning 6-DoF Grasping and Pick-Place Using Attention FocusPreprint
Gualtieri, M., & Platt, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06134 · Cited by 33
Fuzzy and Crisp Representations of Real-Valued Input for Learning Classifier Systems
Bonarini, A., Bonacina, C., & Matteucci, M. (2000) · Lecture notes in computer science · doi:10.1007/3-540-45027-0_5 · Cited by 32
Interactionist-expectative view on agency and learning
Stojanov, G., Bozinovski, S., & Trajkovski, G. P. (1997) · Mathematics and Computers in Simulation · doi:10.1016/s0378-4754(97)00057-8 · Cited by 31
Evolution of Cooperative Problem Solving in an Artificial Economy
Baum, E. B., & Đurđanović, I. (2000) · Neural Computation · doi:10.1162/089976600300014700 · Cited by 30
Gaze-Based Dual Resolution Deep Imitation Learning for High-Precision Dexterous Robot Manipulation
Kim, H., Ohmura, Y., & Kuniyoshi, Y. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3059619 · Cited by 29
Logical Markov Decision Programs and the Convergence of Logical TD(λ)
Kristian Kersting, B., & Raedt, L. D. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30109-7_16 · Cited by 29
A Hybrid Architecture for Situated Learning of Reactive Sequential Decision Making
Sun, R., Peterson, T. S., & Merrill, E. C. (1999) · Applied Intelligence · doi:10.1023/a:1008332731824 · Cited by 28
Bonarini, A. (2001) · Proceedings of the IEEE · doi:10.1109/5.949488 · Cited by 29
The Thing That We Tried Didn't Work Very Well : Deictic Representation in Reinforcement LearningPreprint
Finney, S. J., Gardiol, N. H., Kaelbling, L. P., & Oates, T. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.0567 · Cited by 29
Space is a latent sequence: A theory of the hippocampus
Raju, R. V., Guntupalli, J. S., Zhou, G., Wendelken, C., Lázaro-Gredilla, M., & George, D. (2024) · Science Advances · doi:10.1126/sciadv.adm8470 · Cited by 25
Planning with Neural Networks and Reinforcement Learning
Baldassarre, G. (2003) · Cited by 28
Silvestri, C., Motro, R., Maurin, B., & Dresp, B. (2010) · Design Studies · doi:10.1016/j.destud.2010.03.001 · Cited by 25
Learning Factored Representations for Partially Observable Markov Decision Processes
Sallans, B. (1999) · Cited by 26
Introduction: The Challenge of Reinforcement Learning
Sutton, R. S. (1992) · Machine Learning · doi:10.1023/a:1022620604568 · Cited by 26
Learning in Worlds with Objects
Kaelbling, L. P., Oates, T., Hern, N., & Finney, S. J. (2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_100258 · Cited by 25
A Comprehensive Study of Big Data Machine Learning Approaches and Challenges
Singh, N., Singh, D. P., & Pant, B. (2017) · International Conference on Next Generation Computing and Information Systems (ICNGCIS) · doi:10.1109/icngcis.2017.14 · Cited by 24
Trahanias, P. E., Lourakis, M., Argyros, A., & Orphanoudakis, S. C. (2002) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.1997.614311 · Cited by 24
Machine learning for fast development of advanced energy materials
Farhadi, B., You, J., Zheng, D., Liu, L., Wu, S., Li, J., Li, Z., Wang, K., & Liu, S. F. (2023) · Next Materials · doi:10.1016/j.nxmate.2023.100025 · Cited by 26
Closed-Loop Learning of Visual Control Policies
Jodogne, S. R., & Piater, J. H. (2007) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2110 · Cited by 23
Q-Learning with adaptive state segmentation (QLASS)
Murao, H., & Kitamura, S. (2002) · IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA) · doi:10.1109/cira.1997.613856 · Cited by 22
An inductive logic programming approach to statistical relational learning
Kersting, K. (2006) · Cited by 22
A learning rule that explains how rewards teach attention
Rombouts, J. O., Bohté, S. M., Martínez-Trujillo, J., & Roelfsema, P. R. (2015) · Visual Cognition · doi:10.1080/13506285.2015.1010462 · Cited by 22
Otto, A. R., Gureckis, T. M., Markman, A. B., & Love, B. C. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.5.957 · Cited by 21
Characterizing reinforcement learning methods through parameterized learning problems
Kalyanakrishnan, S., & Stone, P. (2011) · Machine Learning · doi:10.1007/s10994-011-5251-x · Cited by 20
van Hasselt, H. P., Madjiheurem, S., Hessel, M., Silver, D., Barreto, A. D. M. S., & Borsa, D. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i11.17200 · Cited by 10
Selecting treatment strategies with dynamic limited-memory influence diagrams
van Gerven, M. A. J., Díez, F. J., Taal, B. G., & Lucas, P. (2007) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2007.04.004 · Cited by 21
Utile distinction hidden Markov models
Wierstra, D., & Wiering, M. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015346 · Cited by 21
Learning Navigational Behaviors Using a Predictive Sparse Distributed Memory
Rao, R. P. N., & Fuentes, O. (1996) · The MIT Press eBooks · doi:10.7551/mitpress/3118.003.0046 · Cited by 21
Empirical analysis of an on-line adaptive system using a mixture of Bayesian networks
Kitakoshi, D., Shioya, H., & Nakano, R. (2010) · Information Sciences · doi:10.1016/j.ins.2010.04.001 · Cited by 19
Logical Markov Decision Programs
Kristian Kersting, B., & Raedt, L. D. (2003) · Cited by 18
Frommberger, L. (2008) · International Journal of Artificial Intelligence Tools · doi:10.1142/s021821300800400x · Cited by 18
An approach to the design of reinforcement functions in real world, agent-based applications
Bonarini, A., Bonacina, C., & Matteucci, M. (2001) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/3477.931510 · Cited by 17
Is ChatGPT the way toward artificial general intelligence
Emmert‐Streib, F. (2024) · Discover Artificial Intelligence · doi:10.1007/s44163-024-00126-3 · Cited by 17
Metadata
- Authors
- Whitehead, S. D., & Ballard, D. H.
- Affiliations
- University of Rochester (US)
- Venue
- Machine Learning
- Published
- 1991-07
- Volume
- 7
- Issue
- 1
- Pages
- 45-83
- ISSN
- 0885-6125 (ISSN-L)
- 1573-0565
- OpenAlex
- W2052117683
- Cited by
- 300
- References
- 64
- Citations by year
- 2026: 6 · 2025: 4 · 2024: 6 · 2023: 6 · 2022: 3 · 2021: 14 · 2020: 4 · 2019: 7 · 2018: 7 · 2017: 11 · 2016: 6 · 2015: 3 · 2014: 4 · 2013: 10 · 2012: 7
- FWCI
- 12.15
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Machine Learning (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- active perception, perceptual aliasing, reinforcement learning, adaptive control architecture, sensorimotor system, visual attention, decision-making systems, internal representations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review