An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity
Open accessAbstract
To efficiently learn from feedback, cortical networks need to update synaptic weights on multiple levels of cortical hierarchy. An effective and well-known algorithm for computing such changes in synaptic weights is the error backpropagation algorithm. However, in this algorithm, the change in synaptic weights is a complex function of weights and activities of neurons not directly connected with the synapse being modified, whereas the changes in biological synapses are determined only by the activity of presynaptic and postsynaptic neurons. Several models have been proposed that approximate the backpropagation algorithm with local synaptic plasticity, but these models require complex external control over the network or relatively complex plasticity rules. Here we show that a network developed in the predictive coding framework can efficiently perform supervised learning fully autonomously, employing only simple local Hebbian plasticity. Furthermore, for certain parameters, the weight change in the predictive coding model converges to that of the backpropagation algorithm. This suggests that it is possible for cortical networks with simple Hebbian synaptic plasticity to implement efficient learning algorithms in which synapses in areas on multiple levels of hierarchy are modified to minimize the error on the output.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The large majority of predictive coding networks developed for machine-learning tasks fix precisions to the identity throughout training, effectively assuming homoscedastic noise and reducing the free energy to a sum of unweighted squared errors.
“The large majority of PCNs developed for machine-learning tasks [Whittington and Bogacz, 2017, Salvatori et al., 2021, 2024, Millidge et al., 2022, Pinchetti et al., 2025, Oliviers et al., 2025, Song et al., 2024] fix precisions to the identity throughout training, effectively assuming homoscedastic noise and reducing the free energy to a sum of unweighted squared errors.”
From Closed-form predictive coding via hierarchical Gaussian filters (2026) · cites this paper as Whittington and Bogacz, 2017
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredDeep predictive coding implementations almost universally fix the precision matrix to the identity and infer activations by iterative gradient descent.
“deep PC implementations almost universally fix the precision matrix to the identity and infer activations by iterative gradient descent.”
From Closed-form predictive coding via hierarchical Gaussian filters (2026) · cites this paper as Whittington and Bogacz, 2017
In Models > Predictive Coding for Supervised Learning (page unavailable), the paper describes inferring node values by repeatedly changing them along the gradient of a log-probability objective—equivalent to gradient descent on its negative. In Results > Relationship between the Models (page unavailable), it considers setting all variance parameters to 1. It does not establish that deep predictive coding implementations almost universally use an identity precision matrix; its own simulations also use a variance parameter of 100. (LLM, conf=0.87)
Experimental — accuracy not yet measuredIn iterative inference learning, posterior means are obtained by gradient descent run for T approximately 32-128 steps until equilibrium.
“In iterative inference learning, posterior means are obtained by gradient descent μℓ = -∂F/∂µ ℓ run for T ≈ 32-128 steps until equilibrium [Song et al., 2024, Whittington and Bogacz, 2017, Salvatori et al., 2024, Innocenti et al., 2024].”
From Closed-form predictive coding via hierarchical Gaussian filters (2026) · cites this paper as Whittington and Bogacz, 2017
The Inference subsection (page unavailable) describes updating the first moments of approximate distributions along the objective-function gradient, and the model description says updates continue until a steady state. It does not support the stated 32–128 steps: Results > Performance on More Complex Learning Tasks (page unavailable) says the simulations used 20 steps per inference stage and that more steps would have allowed convergence. The paper formulates its update as maximizing a log-probability objective, rather than using the quoted negative-gradient notation. (LLM, conf=0.90)
Experimental — accuracy not yet measuredUnder appropriate conditions, predictive coding recovers the gradients computed by backpropagation through purely local message passing.
“Under appropriate conditions, PC recovers the gradients computed by BP through purely local message passing [Whittington and Bogacz, 2017, Millidge et al., 2022, Song et al., 2024], making it a leading candidate for the next generation of energy-efficient, brain-inspired algorithms.”
From Closed-form predictive coding via hierarchical Gaussian filters (2026) · cites this paper as Whittington and Bogacz, 2017
Eq. (3) is the standard Hebbian rule used in predictive coding networks with fixed identity precision.
“Eq. ( 3) is the standard rule used in predictive coding networks with fixed identity precision [Whittington andBogacz, 2017, Song et al., 2024].”
From Closed-form predictive coding via hierarchical Gaussian filters (2026) · cites this paper as Whittington and Bogacz, 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · see citing works
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · see citing works
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 2× · see citing works
“Predictive coding has been proposed as a general framework for describing computations in the neocortex (Friston, 2010).”
BackgroundRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 6× · see citing works
“…feature of the predictive coding models is that they include additional nodes encoding the difference between the activity on a given level and that predicted by the higher level, and that these prediction errors are propagated through the network (Rao & Ballard, 1999; Friston, 2005).”
BackgroundMethodInfluentialMcClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256 · see citing works
A learning algorithm for Boltzmann machines
Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_7 · Cited by 3,402 · see citing works
Canonical Microcircuits for Predictive Coding
Bastos, A. M., Usrey, W. M., Adams, R. A., Mangun, G. R., Fries, P., & Friston, K. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.038 · Cited by 2,421 · cited 1× · see citing works
“The proposed predictive coding models are still quite abstract, and it is important to investigate if different linear or nonlinear nodes can be mapped on particular anatomically defined neurons within a cortical microcircuit (Bastos et al., 2012).”
BackgroundAction and behavior: a free-energy formulation
Friston, K. J., Daunizeau, J., Kilner, J., & Kiebel, S. J. (2010) · Biological Cybernetics · doi:10.1007/s00422-010-0364-z · Cited by 866 · cited 1× · see citing works
“It has been shown in the past how networks in the predictive coding framework can perform unsupervised learning, attentional modulations, and action selection (Rao & Ballard, 1999; Feldman & Friston, 2010; Friston, Daunizeau, Kilner, & Kiebel, 2010).”
BackgroundNeural repetition suppression reflects fulfilled perceptual expectations
Summerfield, C., Trittschuh, E. H., Monti, J. M., Mesulam, M.-M., & Egner, T. (2008) · Nature Neuroscience · doi:10.1038/nn.2163 · Cited by 692 · see citing works
Flagged first, then by citations
Research works printed but not linked: 36
- Balduzzi D Vanchinathan H Buhmann J Kickback cuts backprop’s red-tape: Biologically plausible credit assignment in neural networks arXiv:1411.6191v1 2014 · arXiv
- Barto A Jordan M Gradient following without back-propagation in layered networks Proceedings of the 1st Annual International Conference on Neural Networks 1987 2 629 636 Piscataway, NJ
- Bell AH Summerfield C Morin EL Malecek NJ Ungerleider LG Encoding of stimulus probability in macaque inferior temporal cortex Current Biology 2016 26 17 2280 27524483 PMC5021632
- Bengio Y How auto-encoders could provide credit assignment in deep networks via target propagation arXiv:1407.7906 2014 · arXiv
- Bengio Y Fischer A Early inference in energy-based models approximates back-propagation arXiv:1510.02777 2015 · arXiv
- Bengio Y Lee D-H Bornschein J Lin Z Towards biologically plausible deep learning arXiv:1502.04156 2015 · arXiv
- Bogacz R A tutorial on the free-energy framework for modelling perception and learning Journal of Mathematical Psychology 2017 76 198 211 28298703 PMC5341759
- Bogacz R Markowska-Kaczmar U Kozik A Blinking artefact recognition in EEG signal using artificial neural network Proceedings of 4th Conference on Neural Networks and Their Applications 1999 502 507 Politechnika Czestochowska
- Bogacz R Moraud EM Abdi A Magill PJ Baufreton J Properties of neurons in external globus pallidus can support optimal action selection PLoS Comput Biol 2016 12 7 e1005004 27389780 PMC4936724
- Brown MW Aggleton JP Recognition memory: What are the roles of the perirhinal cortex and hippocampus? Nature Reviews Neuroscience 2001 2 1 51 61 11253359
- Crick F The recent excitement about neural networks Nature 1989 337 129 132 2911347
- Dayan P Hinton GE Neal RM Zemel RS The Helmholtz machine Neural Computation 1995 7 5 889 904 7584891
- Feldman H Friston K Attention, uncertainty, and free-energy Frontiers in Human Neuroscience 2010 4 215 21160551 PMC3001758
- Fiser A Mahringer D Oyibo HK Petersen AV Leinweber M Keller GB Experience-dependent spatial expectations in mouse visual cortex Nature Neuroscience 2016 19 1658 1664 27618309
- Friston K Learning and inference in the brain Neural Networks 2003 16 1325 1352 14622888
- Friston K A theory of cortical responses Philosophical Transactions of the Royal Society B 2005 360 815 836 PMC1569488 15937014
- Hinton GE McClelland JL Learning representations by recirculation Anderson DZ Neural information processing systems 1988 358 366 New York American Institute of Physics
- Hinton GE Osindero S Teh Y-W A fast learning algorithm for deep belief nets Neural Computation 2006 18 7 1527 1554 16764513
- Hyvarinen A Regression using independent component analysis, and its connection to multi-layer perceptrons Proceedings of the 9th International Conference on Artificial Neural Networks 1999 491 496 Stevenage, UK IEE
- Kingma D Ba J Adam: A method for stochastic optimization arXiv:1412.6980 2014 · arXiv
- Larochelle H Bengio Y Towards biologically plausible deep learning Proceedings of the 25th International Conference on Machine Learning 2008 536 543 New York ACM
- Lillicrap TP Cownden D Tweed DB Akerman CJ Random synaptic feedback weights support error backpropagation for deep learning Nature Communications 2016 7 13276 PMC5105169 27824044
- Mazzoni P Andersen RA Jordan MI A more biologically plausibile learning rule for neural networks Proc Natl Acad Sci USA 1991 88 4433 4437 1903542 PMC51674
- Miller LL Desimone R The representation of stimulus familiarity in anterior inferior temporal cortex Journal of Neurophysiology 1993 69 6 1918 1929 8350131
- Mizuseki K Buzsáki G Preconfigured, skewed distribution of firing rates in the hippocampus and entorhinal cortex Cell Reports 2013 4 5 1010 1021 23994479 PMC3804159
- O’Reilly RC Munakata Y Computational explorations in cognitive neuroscience 2000 Cambridge, MA MIT Press
- Plaut DC McClelland JL Seidenberg MS Patterson K Understanding normal and impaired word reading: Computational principles in quasi-regular domains Psychological Review 1996 103 56 115 8650300
- Scellier B Bengio Y Towards a biologically plausible backprop arXiv:1602.05179 2016 · arXiv
- Seidenberg MS McClelland JL A distributed, developmental model of word recognition and naming Psychological Review 1989 96 523 568 2798649
- Seung HS Learning in spiking neural networks by reinforcement of stochastic synaptic transmission Neuron 2003 40 1063 1073 14687542
- Spratling MW Reconciling predictive coding and biased competition models of cortical function Frontiers in Computational Neuroscience 2008 2 4 18978957 PMC2576514
- Srivastava N Salakhutdinov R Multimodal learning with deep boltzmann machines Pereira F Burges CJC Bottou L Weinberger KQ Advances in neural information processing systems 2012 25 2222 2230 Red Hook, NY Curran
- Summerfield C Egner T Greene M Koechlin E Mangels J Hirsch J Predictive codes for forthcoming perception in the frontal cortex Science 2006 314 1311 1314 17124325
- Werfel J Xiew X Seung HS Learning curves for stochastic gradient descent in linear feedforward networks Neural Computation 2005 17 2699 2718 16212768
- Whittington JC Bogacz R Learning in cortical networks through error back-propagation bioRxiv 2015 035451
- Zmarz P Keller GB Mismatch receptive fields in mouse visual cortex Neuron 2016 92 4 766 772 27974161
Cited by
Sorted by citations
Neuroscience-Inspired Artificial Intelligence
Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.011 · Cited by 1,557 · see citing works
A deep learning framework for neuroscience
Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0520-2 · Cited by 1,027 · see citing works
Self-citationCognitive computational neuroscience
Kriegeskorte, N., & Douglas, P. K. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0210-5 · Cited by 535 · see citing works
If deep learning is the answer, what is the question?
Saxe, A., Nelli, S., & Summerfield, C. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-00395-8 · Cited by 397 · see citing works
Theories of Error Back-Propagation in the Brain
Whittington, J. C. R., & Bogacz, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.005 · Cited by 420 · see citing works
Self-citationThe graphical brain: Belief propagation and active inference
Friston, K. J., Parr, T., & de Vries, B. (2017) · Network Neuroscience · doi:10.1162/netn_a_00018 · Cited by 415 · see citing works
A hierarchy of linguistic predictions during natural language comprehension
Heilbron, M., Armeni, K., Schoffelen, J.-M., Hagoort, P., & de Lange, F. P. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2201968119 · Cited by 343 · see citing works
Deep neural networks in computational neuroscience
Kietzmann, T. C., McClure, P., & Kriegeskorte, N. (2019) · Oxford Research Encyclopedia of Neuroscience · doi:10.1093/acrefore/9780190264086.013.46 · Cited by 186 · see citing works
Deep active inference as variational policy gradients
Millidge, B. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102348 · Cited by 115 · see citing works
Predictive Coding: a Theoretical and Experimental Review
Millidge, B., Seth, A., & Buckley, C. L. (2021) · arXiv · doi:10.48550/arxiv.2107.12979 · Cited by 106 · see citing works
Predictive Coding Approximates Backprop Along Arbitrary Computation Graphs
Millidge, B., Tschantz, A., & Buckley, C. L. (2022) · Neural Computation · doi:10.1162/neco_a_01497 · Cited by 95 · see citing works
Inferring neural activity before plasticity as a foundation for learning beyond backpropagation
Song, Y., Millidge, B., Salvatori, T., Lukasiewicz, T., Xu, Z., & Bogacz, R. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01514-1 · Cited by 85 · see citing works
Self-citationPredictive Coding: Towards a Future of Deep Learning beyond Backpropagation?
Millidge, B., Salvatori, T., Song, Y., Bogacz, R., & Lukasiewicz, T. (2022) · arXiv · doi:10.48550/arxiv.2202.09467 · Cited by 31 · see citing works
Self-citationBrucklacher, M., Bohté, S. M., Mejias, J. F., & Pennartz, C. M. A. (2023) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2023.1207361 · Cited by 15 · see citing works
“…as an implicit objective function, upon which the inference steps conduct an approximate gradient descent taking into account only the sign and not value of the derivative of the activation function, unlike (Whittington and Bogacz, 2017), and upon which learning conducts a precise gradient descent.”
BackgroundMethodInfluentialPredictive coding with spiking neurons and feedforward gist signaling
Lee, K., Dora, S., Mejias, J. F., Bohte, S. M., & Pennartz, C. M. A. (2024) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2024.1338280 · Cited by 14 · see citing works
A predictive-coding network that is both discriminative and generative
Sun, W., & Orchard, J. (2020) · Neural Computation · doi:10.1162/neco_a_01311 · Cited by 11 · see citing works
A survey on neuro-mimetic deep learning via predictive coding
Salvatori, T., Mali, A., Buckley, C. L., Lukasiewicz, T., Rao, R. P. N., Friston, K., & Ororbia, A. (2026) · Neural Networks · doi:10.1016/j.neunet.2025.108161 · Cited by 2 · see citing works
The generalized Hierarchical Gaussian Filter
Weber, L. A., Waade, P. T., Legrand, N., Møller, A. H., Stephan, K. E., & Mathys, C. (2026) · eLife · doi:10.7554/elife.110174
Closed-form predictive coding via hierarchical Gaussian filters
Baskakovs, A., Estebe, S., Enevoldsen, K., Nielbo, K., Mathys, C., & Legrand, N. (2026) · arXiv · doi:10.48550/arxiv.2605.20293
“We accordingly benchmark against the canonical PCN Whittington and Bogacz [2017] and BP, which provide the principled baselines for the comparisons made here.”
Influential
- Authors
- Whittington, J. C. R., & Bogacz, R.
- Affiliations
- John Radcliffe Hospital (GB)
- MRC Brain Network Dynamics Unit, University of Oxford, Oxford OX1 3TH, U.K., and Nuffield Department of Clinical Neurosciences, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, U.K. rafal.bogacz@ndcn.ox.ac.uk
- Corresponding author
- Whittington, James C. R.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2017-05-01
- Volume
- 29
- Issue
- 5
- Pages
- 1229-1262
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- PMC
- PMC5467749
- OpenAlex
- W2604086949
- Cited by
- 278
- References
- 10
- Citations by year
- 2026: 30 · 2025: 36 · 2024: 41 · 2023: 35 · 2022: 31 · 2021: 41 · 2020: 28 · 2019: 15 · 2018: 15 · 2017: 6
- FWCI
- 10.44
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Neural Computation (publishedVersion, cc-by-nc)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion, cc-by)
- PubMed Central (acceptedVersion, cc-by)
- Funders
- Medical Research Council (MC_UU_12024/5)
- Engineering and Physical Sciences Research Council
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- backpropagation, predictive coding, Hebbian synaptic plasticity, cortical hierarchy, synaptic weight update
- MeSH terms
- Algorithms, Cerebral Cortex, Humans, Learning, Models, Neurological, Neuronal Plasticity, Neurons, Neural Networks, Computer, Feedback, Physiological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
