CoChat
Sign Up
Cited by 1,450Complex Network Analysis TechniquesNeural Networks and ApplicationsMental Health Research Topics

Toward a modern theory of adaptive networks: Expectation and prediction.

Sutton, Richard S., Barto, Andrew G.·1981·Psychological Review
DOI 10.1037/0033-295x.88.2.135
Open this paper in CoChat — free account

Abstract

Many adaptive neural network theories are based on neuronlike adaptive elements that can behave as single unit analogs of associative conditioning. In this article we develop a similar adaptive element, but one which is more closely in accord with the facts of animal learning theory than elements commonly studied in adaptive network research. We suggest that an essential feature of classical conditioning that has been largely overlooked by adaptive network theorists is its predictive nature. The adaptive element we present learns to increase its response rate in anticipation of increased stimulation, producing a conditioned response before the occurrence of the unconditioned stimulus. The element also is in strong agreement with the behavioral data regarding the effects of stimulus context, since it is a temporally refined extension of the Rescorla-Wagner model. We show by computer simulation that the element becomes sensitive to the most reliable, nonredundant, and earliest predictors of reinforcement. We also point out that the model solves many of the stability and saturation problems encountered in network simulations. Finally, we discuss our model in light of recent advances in the physiology and biochemistry of synaptic mechanisms. One way to bridge the gap between behavioral and neural views of learning is to postulate neural analogs of behavioral modification paradigms. Hebb's suggestion that when a cell A repeatedly and persistently takes part in firing another cell B, then A's efficiency in firing B is increased, is the most familiar of these postulates (Hebb, 1949). This rule for synaptic plasticity is a neural analog of associative conditioning and continues to exert a powerful influence on theoretical and experimental research in learning and memory. Neural network models designed to explore the behavioral possibilities of modifiable structures typically em-

Integrity signals

Registry

Crossref · 10.1037/0033-295x.88.2.135

Cites

71 of 71 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • We obtained similar results with a more biologically plausible stochastic gradient learning approach using weight perturbations modified from ref. 57.

    “To solve Eq. 1 , we used the nonbiological MATLABleft division operation for matrices, but we obtained similar results with a more biologically plausible stochastic gradient learning approach using weight perturbations modified from ref. 57, i.e., ( x i , y i ) pairs, where x i denotes the i th image in the training set and y i is its associated label (animal or non-animal).”

    From A feedforward architecture accounts for rapid categorization (2007) · cites this paper as [57]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Sutton, R. S., & Barto, A. G.
Affiliations
University of Massachusetts Amherst (US)
ORCID
Venue
Psychological Review
Publisher
American Psychological Association (APA)
Published
1981
Volume
88
Issue
2
Pages
135-170
ISSN
  • 0033-295X (ISSN-L)
  • 1939-1471
DOI
10.1037/0033-295x.88.2.135
OpenAlex
W2040598998
Cited by
1,450
References
71
Citations by year
2026: 23 · 2025: 26 · 2024: 45 · 2023: 35 · 2022: 32 · 2021: 32 · 2020: 28 · 2019: 30 · 2018: 35 · 2017: 34 · 2016: 31 · 2015: 36 · 2014: 35 · 2013: 35 · 2012: 43
FWCI
8.73
Type
journal article
Language
en
Open access
closed
Locations
  • Psychological Review (publishedVersion)
  • PubMed (publishedVersion)
  • ScholarWorks@UMassAmherst (University of Massachusetts Amherst) (submittedVersion)
  • Scholarworks (University of Massachusetts Amherst) (submittedVersion)
Topics
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
Keywords
adaptive networks, expectations, prediction
MeSH terms
Animals, Association Learning, Conditioning, Psychological, Conditioning, Classical, Models, Neurological, Nervous System Physiological Phenomena, Neurons, Perception, Synapses

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review