CoChat
Sign Up
Cited by 352Neural Networks and ApplicationsNeural dynamics and brain functionAdvanced Memory and Neural Computing

Biologically plausible error-driven learning using local activation differences: the generalized recirculation algorithm

O'Reilly, Randall C.·1996·Neural Computation
DOI 10.1162/neco.1996.8.5.895
Open this paper in CoChat — free account

Abstract

The error backpropagation learning algorithm (BP) is generally considered biologically implausible because it does not use locally available, activation-based variables. A version of BP that can be computed locally using bidirectional activation recirculation (Hinton and McClelland 1988) instead of backpropagated error derivatives is more biologically plausible. This paper presents a generalized version of the recirculation algorithm (GeneRec), which overcomes several limitations of the earlier algorithm by using a generic recurrent network with sigmoidal units that can learn arbitrary input/output mappings. However, the contrastive Hebbian learning algorithm (CHL, also known as DBM or mean field learning) also uses local variables to perform error-driven learning in a sigmoidal recurrent network. CHL was derived in a stochastic framework (the Boltzmann machine), but has been extended to the deterministic case in various ways, all of which rely on problematic approximations and assumptions, leading some to conclude that it is fundamentally flawed. This paper shows that CHL can be derived instead from within the BP framework via the GeneRec algorithm. CHL is a symmetry-preserving version of GeneRec that uses a simple approximation to the midpoint or second-order accurate Runge-Kutta method of numerical integration, which explains the generally faster learning speed of CHL compared to BI. Thus, all known fully general error-driven learning algorithms that use local activation-based variables in deterministic networks can be considered variations of the GeneRec algorithm (and indirectly, of the backpropagation algorithm). GeneRec therefore provides a promising framework for thinking about how the brain might perform error-driven learning. To further this goal, an explicit biological mechanism is proposed that would be capable of implementing GeneRec-style learning. This mechanism is consistent with available evidence regarding synaptic modification in neurons in the neocortex and hippocampus, and makes further predictions.

Integrity signals

Registry

Crossref · 10.1162/neco.1996.8.5.895

Cites

54 of 54 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • There are now many proposed biologically plausible mechanisms by which a neural circuit could implement optimization algorithms that, like backpropagation, can efficiently make use of the gradient.

    “Contrary to what was once a common assumption, there are now many proposed “biologically plausible” mechanisms by which a neural circuit could implement optimization algorithms that, like backpropagation, can efficiently make use of the gradient.”

    From Toward an integration of deep learning and neuroscience (2016) · cites this paper as O'Reilly, 1996

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
O'Reilly, R. C.
Affiliations
Department of Psychology, Carnegie Mellon University, Pittsburgh, PA 15213 USA
Venue
Neural Computation
Publisher
MIT Press - Journals
Published
1996-07
Volume
8
Issue
5
Pages
895-938
ISSN
  • 0899-7667 (ISSN-L)
  • 1530-888X
DOI
10.1162/neco.1996.8.5.895
OpenAlex
W2110278466
Cited by
352
References
54
Citations by year
2026: 6 · 2025: 10 · 2024: 15 · 2023: 28 · 2022: 22 · 2021: 17 · 2020: 9 · 2019: 14 · 2018: 9 · 2017: 17 · 2016: 14 · 2015: 8 · 2014: 17 · 2013: 11 · 2012: 9
FWCI
2.39
Type
journal article
Language
en
Open access
closed
Locations
Neural Computation (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
Keywords
biologically plausible learning, error-based learning, backpropagation, recurrent neural networks, synaptic plasticity, neocortex, hippocampus

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review