Predictive Coding Approximates Backprop Along Arbitrary Computation Graphs
Open accessAbstract
Backpropagation of error (backprop) is a powerful algorithm for training machine learning architectures through end-to-end differentiation. Recently it has been shown that backprop in multilayer perceptrons (MLPs) can be approximated using predictive coding, a biologically plausible process theory of cortical computation that relies solely on local and Hebbian updates. The power of backprop, however, lies not in its instantiation in MLPs but in the concept of automatic differentiation, which allows for the optimization of any differentiable program expressed as a computation graph. Here, we demonstrate that predictive coding converges asymptotically (and in practice, rapidly) to exact backprop gradients on arbitrary computation graphs using only local learning rules. We apply this result to develop a straightforward strategy to translate core machine learning architectures into their predictive coding equivalents. We construct predictive coding convolutional neural networks, recurrent neural networks, and the more complex long short-term memory, which include a nonlayer-like branching internal graph structure and multiplicative interactions. Our models perform equivalently to backprop on challenging machine learning benchmarks while using only local and (mostly) Hebbian plasticity. Our method raises the potential that standard machine learning algorithms could in principle be directly implemented in neural circuitry and may also contribute to the development of completely distributed neuromorphic architectures.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Under appropriate conditions, predictive coding recovers the gradients computed by backpropagation through purely local message passing.
“Under appropriate conditions, PC recovers the gradients computed by BP through purely local message passing [Whittington and Bogacz, 2017, Millidge et al., 2022, Song et al., 2024], making it a leading candidate for the next generation of energy-efficient, brain-inspired algorithms.”
From Closed-form predictive coding via hierarchical Gaussian filters (2026) · cites this paper as Millidge et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1993.341010 · Cited by 14,474 · see citing works
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881 · see citing works
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653 · see citing works
(2017) · Cited by 11,625 · see citing works
Advances in Neural Information Processing Systems 14
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 9,184 · see citing works
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · see citing works
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 1× · see citing works
“…a single mechanism that accounts for diverse perceptual phenomena such as repetition-suppression (Auksztulewicz & Friston, 2016), end-stopping (Rao & Ballard, 1999), bistable perception (Hohwy, Roepstorff, & Friston, 2008; Weilnhammer, Stuke, Hesselmann, Sterzer, & Schmack, 2017) and illusory…”
Buzsáki, G. (2006) · doi:10.1093/acprof:oso/9780195301069.001.0001 · Cited by 4,873 · see citing works
A theory of cortical responses.
Friston, K. (2005) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2005.1622 · Cited by 4,578 · see citing works
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369 · see citing works
A million spiking-neuron integrated circuit with a scalable communication network and interface
Merolla, P. A., Arthur, J. V., Alvarez-Icaza, R., Cassidy, A. S., Sawada, J., Akopyan, F., Jackson, B. L., Imam, N., Chen, G., Nakamura, Y., et al. (2014) · Science · doi:10.1126/science.1254642 · Cited by 4,137 · see citing works
Variational Inference: A Review for Statisticians
Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1285773 · Cited by 3,974 · see citing works
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning
Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G. N., Cao, Y., Choday, S. H., Dimou, G. D., Joshi, P., Imam, N., Jain, S., et al. (2018) · IEEE Micro · doi:10.1109/mm.2018.112130359 · Cited by 3,846 · see citing works
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455 · see citing works
Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems
Harville, D. A. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10480998 · Cited by 2,667 · see citing works
Canonical Microcircuits for Predictive Coding
Bastos, A. M., Usrey, W. M., Adams, R. A., Mangun, G. R., Fries, P., & Friston, K. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.038 · Cited by 2,421 · cited 1× · see citing works
“(Bastos et al., 2012; Kanai et al., 2015; Shipp, 2016; Spratling, 2008).”
The distinct modes of vision offered by feedforward and recurrent processing
Lamme, V. A. F., & Roelfsema, P. R. (2000) · Trends in Neurosciences · doi:10.1016/s0166-2236(00)01657-x · Cited by 2,338 · cited 1× · see citing works
“Neurally, this could correspond to an initial feedforward sweep, for which there is substantial evidence in the brain (Bullier, 2001; Hung, Kreiman, Poggio, & DiCarlo, 2005; Lamme & Roelfsema, 2000; Serre, Oliva, & Poggio, 2007; VanRullen, 2007).”
BackgroundNeural ordinary differential equations
Chen, R. T. Q., Rubanova, Y., Bettencourt, J., & Duvenaud, D. (2018) · arXiv · doi:10.48550/arxiv.1806.07366 · Cited by 2,097 · see citing works
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics 2016
Murray, I., & Graham, M. (2016) · Cited by 2,079 · see citing works
Variational Inference: A Review for Statisticians
Blei, D. M., Kucukelbir, A., & McAuliffe, J. (2017) · Figshare · doi:10.6084/m9.figshare.5203696 · Cited by 2,048 · see citing works
Automatic Differentiation in Machine Learning: a Survey
Baydin, A. G., Pearlmutter, B. A., Radul, A., & Siskind, J. M. (2015) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 2,014 · see citing works
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924 · see citing works
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 1× · see citing works
“Deep learning has seen stunning successes in the last decade in computer vision (Krizhevsky, Sutskever, & Hinton, 2012; Szegedy et al., 2015), natural language processing and translation (Kaplan et al., 2020; Radford et al., 2019; a r X i v : 2006 .”
Background(2003) · Cited by 1,680 · see citing works
Neuroscience-Inspired Artificial Intelligence
Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.011 · Cited by 1,557 · see citing works
Feature Discovery by Competitive Learning*
Rumelhart, D. E., & Zipser, D. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_5 · Cited by 1,379 · see citing works
Furber, S. B., Galluppi, F., Temple, S., & Plana, L. A. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2304638 · Cited by 1,373 · see citing works
Attention, uncertainty, and free-energy
Feldman, H., & Friston, K. J. (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00215 · Cited by 1,365 · see citing works
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · RePEc: Research Papers in Economics · Cited by 1,310 · see citing works
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204 · see citing works
Predictive coding: a fresh view of inhibition in the retina
Srinivasan, M. V., Laughlin, S. B., & Dubs, A. (1982) · Proceedings of the Royal Society of London. Series B. Biological Sciences · doi:10.1098/rspb.1982.0085 · Cited by 1,042 · see citing works
A deep learning framework for neuroscience
Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0520-2 · Cited by 1,027 · see citing works
Integrated model of visual processing
Bullier, J. (2001) · Brain Research Reviews · doi:10.1016/s0165-0173(01)00085-6 · Cited by 1,016 · see citing works
Hierarchical Models in the Brain
Friston, K. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000211 · Cited by 952 · cited 2× · see citing works
“Mathematically, predictive coding corresponds to a variational inference algorithm under a hierarchical Gaussian generative model p ( y 0 . . . y N ) = p ( y 0 ) (Buckley, Kim, McGregor, & Seth, 2017; Friston, 2005, 2008).”
MethodLillicrap, T. P., Santoro, A., Marris, L., Akerman, C. J., & Hinton, G. (2020) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-0277-3 · Cited by 908 · see citing works
Variational free energy and the Laplace approximation
Friston, K., Mattout, J., Trujillo-Barreto, N., Ashburner, J., & Penny, W. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2006.08.035 · Cited by 880 · see citing works
Random synaptic feedback weights support error backpropagation for deep learning
Lillicrap, T., Cownden, D., Tweed, D., & Akerman, C. J. (2016) · Nature Communications · doi:10.1038/ncomms13276 · Cited by 879 · see citing works
A Unifying Review of Linear Gaussian Models
Roweis, S. T., & Zoubin Ghahramani, A. (1999) · Neural Computation · doi:10.1162/089976699300016674 · Cited by 873 · see citing works
Fast Readout of Object Identity from Macaque Inferior Temporal Cortex
Hung, C. P., Kreiman, G., Poggio, T., & DiCarlo, J. J. (2005) · Science · doi:10.1126/science.1117593 · Cited by 854 · see citing works
Deep Predictive Coding Networks for Video Prediction and Unsupervised LearningPreprint
Lotter, W. E., Kreiman, G., & Cox, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08104 · Cited by 805 · see citing works
The recent excitement about neural networks
Crick, F. (1989) · Nature · doi:10.1038/337129a0 · Cited by 724 · see citing works
Learning and inference in the brain
Friston, K. (2003) · Neural Networks · doi:10.1016/j.neunet.2003.06.005 · Cited by 724 · see citing works
Black Box Variational InferencePreprint
Ranganath, R., Gerrish, S., & Blei, D. M. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.0118 · Cited by 646 · see citing works
Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
Lindsay, G. W. (2020) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01544 · Cited by 571 · see citing works
Predictive coding explains binocular rivalry: an epistemological review
Hohwy, J., Roepstorff, A., & Friston, K. (2008) · Cognition · doi:10.1016/j.cognition.2008.05.010 · Cited by 546 · see citing works
Equilibrium Propagation: Bridging the Gap between Energy-Based Models and Backpropagation
Scellier, B., & Bengio, Y. (2017) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2017.00024 · Cited by 504 · see citing works
Kass, R. E., & Steffey, D. (1989) · Journal of the American Statistical Association · doi:10.1080/01621459.1989.10478825 · Cited by 474 · see citing works
Stochastic Gradient Hamiltonian Monte CarloPreprint
Chen, T., Fox, E. B., & Guestrin, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.4102 · Cited by 469 · see citing works
Estimation in Covariance Components Models
Dempster, A. P., Rubin, D. B., & Tsutakawa, R. K. (1981) · Journal of the American Statistical Association · doi:10.1080/01621459.1981.10477653 · Cited by 444 · see citing works
Eliaš, P. (1955) · IEEE Transactions on Information Theory · doi:10.1109/tit.1955.1055126 · Cited by 436 · see citing works
Cerebral hierarchies: predictive processing, precision and the pulvinar
Kanai, R., Komura, Y., Shipp, S., & Friston, K. (2015) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2014.0169 · Cited by 432 · see citing works
Theories of Error Back-Propagation in the Brain
Whittington, J. C. R., & Bogacz, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.005 · Cited by 420 · see citing works
Towards deep learning with segregated dendrites
Guerguiev, J., Lillicrap, T., & Richards, B. A. (2017) · eLife · doi:10.7554/elife.22901 · Cited by 373 · see citing works
The free energy principle for action and perception: A mathematical review
Buckley, C. L., Kim, C. S., McGregor, S., & Seth, A. K. (2017) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2017.09.004 · Cited by 352 · see citing works
Self-citationInformation Processing in The Nervous System
Leibovic, K. N. (1969) · Springer eBooks · doi:10.1007/978-3-662-25549-0 · Cited by 352 · see citing works
Learning in spiking neural networks by reinforcement of stochastic synaptic transmission
Seung, H. S. (2003) · Neuron · doi:10.1016/s0896-6273(03)00761-x · Cited by 340 · see citing works
Neural elements for predictive coding
Shipp, S. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01792 · Cited by 340 · see citing works
Seeing it all: Convolutional network layers map the function of the human visual system
Eickenberg, M., Gramfort, A., Varoquaux, G., & Thirion, B. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.10.001 · Cited by 338 · see citing works
Stochastic Gradient Descent as Approximate Bayesian Inference
Mandt, S., Hoffman, M. D., & Blei, D. M. (2017) · Journal of Machine Learning Research · doi:10.5555/3122009.3208015 · Cited by 337 · see citing works
Information geometry of the EM and em algorithms for neural networks
Амари, Ш.-И. (1995) · Neural Networks · doi:10.1016/0893-6080(95)00003-8 · Cited by 326 · see citing works
Repetition suppression and its contextual determinants in predictive coding
Auksztulewicz, R., & Friston, K. (2016) · Cortex · doi:10.1016/j.cortex.2015.11.024 · Cited by 307 · see citing works
A tutorial on the free-energy framework for modelling perception and learning
Bogacz, R. (2017) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.11.003 · Cited by 283 · see citing works
Whittington, J. C. R., & Bogacz, R. (2017) · Neural Computation · doi:10.1162/neco_a_00949 · Cited by 278 · see citing works
Applications of advances in nonlinear sensitivity analysis
Werbos, P. J. (2005) · Springer eBooks · doi:10.1007/bfb0006203 · Cited by 272 · see citing works
Lee, D., Zhang, S., Fischer, A., & Bengio, Y. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-23528-8_31 · Cited by 225 · see citing works
Backpropagation-decorrelation: online recurrent learning with O(N) complexity
Steil, J. J. (2005) · IEEE International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2004.1380039 · Cited by 206 · see citing works
A Differentiable Physics Engine for Deep Learning in Robotics
Degrave, J., Hermans, M., Dambre, J., & wyffels, F. (2019) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2019.00006 · Cited by 182 · see citing works
Reconciling predictive coding and biased competition models of cortical function
Spratling, M. W. (2008) · Frontiers in Computational Neuroscience · doi:10.3389/neuro.10.004.2008 · Cited by 153 · see citing works
Backpropagation through time and the brain
Lillicrap, T., & Santoro, A. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.01.011 · Cited by 142 · see citing works
How Important Is Weight Symmetry in Backpropagation?
Liao, Q., Leibo, J. Z., & Poggio, T. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10279 · Cited by 136 · see citing works
A predictive coding account of bistable perception - a model-based fMRI study
Weilnhammer, V., Stuke, H., Hesselmann, G., Sterzer, P., & Schmack, K. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005536 · Cited by 120 · see citing works
Ruck, D. W., Rogers, S. K., Kabrisky, M., Maybeck, P. S., & Oxley, M. E. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.141559 · Cited by 117 · see citing works
STDP-Compatible Approximation of Backpropagation in an Energy-Based Model
Bengio, Y., Mesnard, T., Fischer, A., Zhang, S., & Wu, Y. (2017) · Neural Computation · doi:10.1162/neco_a_00934 · Cited by 108 · see citing works
Predictive Coding: a Theoretical and Experimental Review
Millidge, B., Seth, A., & Buckley, C. L. (2021) · arXiv · doi:10.48550/arxiv.2107.12979 · Cited by 106 · see citing works
Self-citationIllusory Motion Reproduced by Deep Neural Networks Trained for Prediction
Watanabe, E., Kitaoka, A., Sakamoto, K., Yasugi, M., & Tanaka, K. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00345 · Cited by 101 · see citing works
The power of the feed-forward sweep
Rufin Vanrullen, A. (2007) · Advances in Cognitive Psychology · doi:10.2478/v10053-008-0022-3 · Cited by 99 · see citing works
Forward-Mode Automatic Differentiation in Julia
Revels, J., Lubin, M., & Papamarkou, T. (2016) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · doi:10.48550/arxiv.1607.07892 · Cited by 99 · see citing works
Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion LimitPreprint
Tzen, B., & Raginsky, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.09883 · Cited by 96 · see citing works
DiffEqFlux.jl - A Julia Library for Neural Differential EquationsPreprint
Rackauckas, C., Innes, M. J., Ma, Y., Bettencourt, J., White, L., & Dixit, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02376 · Cited by 84 · see citing works
Laporte, E., & Tallec, P. L. (2003) · Modeling and simulation in science, engineering & technology · doi:10.1007/978-1-4612-0069-7_5 · Cited by 81 · see citing works
Deep Learning without Weight TransportPreprint
Akrout, M., Wilson, C., Humphreys, P. C., Lillicrap, T., & Tweed, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05391 · Cited by 80 · see citing works
Zygote: A Differentiable Programming System to Bridge Machine Learning and Scientific ComputingPreprint
Innes, M. J., Edelman, A. S., Fischer, K., Rackauckas, C., Saba, E., Shah, V. B., & Tebbutt, W. (2019) · arXiv (Cornell University) · Cited by 61 · see citing works
Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representations
Ororbia, A. G., Mali, A., Giles, C. L., & Kifer, D. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2953622 · Cited by 60 · see citing works
Deep Learning With Asymmetric Connections and Hebbian Updates
Amit, Y. (2019) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2019.00018 · Cited by 38 · see citing works
Early Inference in Energy-Based Models Approximates Back-PropagationPreprint
Bengio, Y., & Fischer, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.02777 · Cited by 37 · see citing works
Path Integral Networks: End-to-End Differentiable Optimal ControlPreprint
Okada, M., Rigazio, L., & Aoshima, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.09597 · Cited by 36 · see citing works
Unbiased Online Recurrent OptimizationPreprint
Ollivier, Y., & Tallec, C. (2017) · HAL · Cited by 31 · see citing works
The Differentiable Cross-Entropy MethodPreprint
Amos, B., & Yarats, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12830 · Cited by 30 · see citing works
Generalization of Equilibrium Propagation to Vector Field DynamicsPreprint
Scellier, B., Goyal, A., Binas, J., Mesnard, T., & Bengio, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04873 · Cited by 26 · see citing works
Training recurrent networks online without backtrackingPreprint
Ollivier, Y., Tallec, C., & Charpiat, G. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.07680 · Cited by 24 · see citing works
Millidge, B. (2019) · doi:10.31234/osf.io/kf6wc · Cited by 21 · see citing works
Self-citationRelaxing the Constraints on Predictive Coding ModelsPreprint
Millidge, B., Tschantz, A., Seth, A. K., & Buckley, C. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01047 · Cited by 17 · see citing works
Self-citationImplementing Predictive Processing and Active Inference: Preliminary Steps and ResultsPreprint
Millidge, B. (2019) · doi:10.31234/osf.io/4hb58 · Cited by 17 · see citing works
Self-citationInvariant recognition drives neural representations of action sequences
Tacchetti, A., Işık, L., & Poggio, T. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005859 · Cited by 16 · see citing works
Predictive Coding Can Do Exact Backpropagation on Convolutional and Recurrent Neural Networks
Salvatori, T., Song, Y., Lukasiewicz, T., Bogacz, R., & Xu, Z. (2021) · arXiv · doi:10.48550/arxiv.2103.03725 · Cited by 16 · see citing works
Neural Kalman FilteringPreprint
Millidge, B., Tschantz, A., Seth, A. K., & Buckley, C. L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10021 · Cited by 14 · see citing works
Self-citationUnbiased Online Recurrent OptimizationPreprint
Tallec, C., & Ollivier, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05043 · Cited by 14 · see citing works
The Extended Kalman Filter is a Natural Gradient Descent in Trajectory SpacePreprint
Ollivier, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00696 · Cited by 10 · see citing works
Real2Sim Transfer using Differentiable Physics
Heiden, E., Millard, D., & Sukhatme, G. S. (2019) · Cited by 7 · see citing works
Pal, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.07198 · Cited by 3 · see citing works
Pal, A. (2020) · JuliaCon Proceedings · doi:10.21105/jcon.00037 · Cited by 1 · see citing works
Flagged first, then by citations
5 of the 113 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Linnainmaa, S. (1970). The representation of the cumulative rounding error of an algorithm as a taylor expansion of the local rounding errors. Master's Thesis (in Finnish), Univ. Helsinki, 6-7.
- Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1(8), 9.
Cited by
Sorted by citations
Hoefler, T., Alistarh, D., Ben-Nun, T., Dryden, N., & Peste, A. (2021) · ETH Zurich · doi:10.3929/ethz-b-000519202 · Cited by 423 · see citing works
A hierarchy of linguistic predictions during natural language comprehension
Heilbron, M., Armeni, K., Schoffelen, J.-M., Hagoort, P., & de Lange, F. P. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2201968119 · Cited by 343 · see citing works
Macpherson, T., Churchland, A. K., Sejnowski, T., DiCarlo, J. J., Kamitani, Y., Takahashi, H., & Hikida, T. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.09.018 · Cited by 157 · see citing works
Predictive Coding: a Theoretical and Experimental Review
Millidge, B., Seth, A., & Buckley, C. L. (2021) · arXiv · doi:10.48550/arxiv.2107.12979 · Cited by 106 · see citing works
Self-citationInferring neural activity before plasticity as a foundation for learning beyond backpropagation
Song, Y., Millidge, B., Salvatori, T., Lukasiewicz, T., Xu, Z., & Bogacz, R. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01514-1 · Cited by 85 · see citing works
Self-citationHow particular is the physics of the free energy principle?
Millidge, B. (2022) · Sussex Research Online (University of Sussex) · Cited by 83 · see citing works
Self-citationA predictive coding model of the N400
Eddine, S. N., Brothers, T. A., Wang, L., Spratling, M., & Kuperberg, G. R. (2024) · Cognition · doi:10.1016/j.cognition.2024.105755 · Cited by 68 · see citing works
Song, Y., Lukasiewicz, T., Xu, Z., & Bogacz, R. (2020) · PubMed · Cited by 60 · see citing works
Hertäg, L., & Clopath, C. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2115699119 · Cited by 55 · see citing works
Hybrid predictive coding: Inferring, fast and slow
Tscshantz, A., Millidge, B., Seth, A. K., & Buckley, C. L. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011280 · Cited by 51 · see citing works
Self-citationPredictive coding feedback results in perceived illusory contours in a recurrent neural network
Pang, Z., O’May, C. B., Choksi, B., & VanRullen, R. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.08.024 · Cited by 44 · see citing works
Predictive coding networks for temporal prediction
Millidge, B., Tang, M., Osanlouy, M., Harper, N. S., & Bogacz, R. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011183 · Cited by 44 · see citing works
Self-citationThe N400 in silico: A review of computational models
Eddine, S. N., Brothers, T. A., & Kuperberg, G. R. (2022) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/bs.plm.2022.03.005 · Cited by 32 · see citing works
Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?
Millidge, B., Salvatori, T., Song, Y., Bogacz, R., & Lukasiewicz, T. (2022) · arXiv · doi:10.48550/arxiv.2202.09467 · Cited by 31 · see citing works
Self-citationPredictive Coding: Towards a Future of Deep Learning beyond Backpropagation?
Millidge, B., Salvatori, T., Song, Y., Bogacz, R., & Lukasiewicz, T. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/774 · Cited by 35 · see citing works
Self-citationRecurrent predictive coding models for associative memory employing covariance learning
Tang, M., Salvatori, T., Millidge, B., Song, Y., Lukasiewicz, T., & Bogacz, R. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010719 · Cited by 31 · see citing works
Self-citationSoftHebb: Bayesian inference in unsupervised Hebbian soft winner-take-all networks
Moraitis, T., Toichkin, D., Journé, A., Chua, Y., & Guo, Q. (2022) · Neuromorphic Computing and Engineering · doi:10.1088/2634-4386/aca710 · Cited by 28 · see citing works
Hierarchical predictive coding in distributed pain circuits
Chen, Z. S. (2023) · Frontiers in Neural Circuits · doi:10.3389/fncir.2023.1073537 · Cited by 34 · see citing works
The free energy principle induces neuromorphic development
Fields, C., Friston, K. J., Glazebrook, J. F., Levin, M. G., & Marcianò, A. (2022) · Neuromorphic Computing and Engineering · doi:10.1088/2634-4386/aca7de · Cited by 29 · see citing works
Suzuki, K., Seth, A. K., & Schwartzman, D. J. (2024) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2023.1159821 · Cited by 25 · see citing works
On the relationship between predictive coding and backpropagation
Rosenbaum, R. (2022) · PLoS ONE · doi:10.1371/journal.pone.0266102 · Cited by 20 · see citing works
State-space modeling in long sequence processing: a survey on recurrence in the transformer era
Tiezzi, M., Casoni, M., Betti, A., Gori, M., & Melacci, S. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108039 · Cited by 21 · see citing works
Reverse Differentiation via Predictive Coding
Salvatori, T., Song, Y., Xu, Z., Lukasiewicz, T., & Bogacz, R. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20788 · Cited by 19 · see citing works
Edwards, D. J., McEnteggart, C., & Barnes‐Holmes, Y. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.745306 · Cited by 16 · see citing works
A method for the ethical analysis of brain-inspired AI
Farisco, M., Baldassarre, G. D., Cartoni, E., Leach, A., Petrovici, M. A., Rosemann, A., Salles, A., Stahl, B. C., & van Albada, S. J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10769-4 · Cited by 19 · see citing works
Predictive Coding Can Do Exact Backpropagation on Convolutional and Recurrent Neural Networks
Salvatori, T., Song, Y., Lukasiewicz, T., Bogacz, R., & Xu, Z. (2021) · arXiv · doi:10.48550/arxiv.2103.03725 · Cited by 16 · see citing works
A Review of Neuroscience-Inspired Machine LearningPreprint
Ororbia, A. G., Mali, A., Kohan, A., Millidge, B., & Salvatori, T. (2024) · doi:10.31219/osf.io/uz8cv · Cited by 19 · see citing works
Self-citationLearning on Arbitrary Graph Topologies via Predictive Coding
Salvatori, T., Pinchetti, L., Millidge, B., Song, Y., Bao, T.-Y., Bogacz, R., & Lukasiewicz, T. (2022) · PubMed · doi:10.48550/arxiv.2201.13180 · Cited by 27 · see citing works
Self-citationMakkeh, A., Graetz, M., Schneider, A. C., Ehrlich, D. A., Priesemann, V., & Wibral, M. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2408125122 · Cited by 14 · see citing works
Relaxing the Constraints on Predictive Coding ModelsPreprint
Millidge, B., Tschantz, A., Seth, A. K., & Buckley, C. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01047 · Cited by 17 · see citing works
Self-citationMonotone Quantifiers Emerge via Iterated Learning
Carcassi, F., Steinert‐Threlkeld, S., & Szymanik, J. (2021) · Cognitive Science · doi:10.1111/cogs.13027 · Cited by 11 · see citing works
Activation Relaxation: A Local Dynamical Approximation to Backpropagation in the BrainPreprint
Millidge, B., Tschantz, A., Seth, A. K., & Buckley, C. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.05359 · Cited by 12 · see citing works
Self-citationAnnabi, L., Pitti, A., & Quoy, M. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.07.016 · Cited by 10 · see citing works
Active Inference and Epistemic Value in Graphical Models
van de Laar, T., Koudahl, M., van Erp, B., & de Vries, B. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.794464 · Cited by 10 · see citing works
Predictive Coding as a Neuromorphic Alternative to Backpropagation: A Critical Evaluation
Zahid, U., Guo, Q., & Fountas, Z. (2023) · Neural Computation · doi:10.1162/neco_a_01620 · Cited by 10 · see citing works
Bayesian Brains and the Rényi Divergence
Sajid, N., Faccio, F., Costa, L. D., Parr, T., Schmidhuber, J., & Friston, K. J. (2022) · Neural Computation · doi:10.1162/neco_a_01484 · Cited by 9 · see citing works
Brain-inspired Predictive Coding Improves the Performance of Machine Challenging Tasks
Lee, J., Jo, J., Lee, B.-H., Lee, J.-H., & Yoon, S. (2022) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2022.1062678 · Cited by 9 · see citing works
Active Vision in Binocular Depth Estimation: A Top-Down Perspective
Priorelli, M., Pezzulo, G., & Stoianov, I. P. (2023) · Biomimetics · doi:10.3390/biomimetics8050445 · Cited by 9 · see citing works
Dynamic planning in hierarchical active inference
Priorelli, M., & Stoianov, I. P. (2025) · Neural Networks · doi:10.1016/j.neunet.2024.107075 · Cited by 9 · see citing works
Goekoop, R., & de Kleijn, R. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105402 · Cited by 10 · see citing works
Feed-Forward Optimization With Delayed Feedback for Neural Network Training
Flügel, K., Coquelin, D., Weiel, M., Debus, C., Streit, A., & Götz, M. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-6585-3_6 · Cited by 8 · see citing works
Fast On-Device Adaptation for Spiking Neural Networks Via Online-Within-Online Meta-Learning
Rosenfeld, B., Rajendran, B., & Simeone, O. (2021) · IEEE Data Science Workshop (DSW) · doi:10.1109/dslw51110.2021.9523405 · Cited by 8 · see citing works
Do Biological Constraints Impair Dendritic Computation?
Jones, I., & Kording, K. P. (2021) · Neuroscience · doi:10.1016/j.neuroscience.2021.07.036 · Cited by 9 · see citing works
Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit AssignmentPreprint
Ororbia, A. G. (2023) · doi:10.31219/osf.io/xukqf · Cited by 10 · see citing works
Biologically plausible local synaptic learning rules robustly implement deep supervised learning
Konishi, M., Igarashi, K. M., & Miura, K. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1160899 · Cited by 7 · see citing works
Inference on the Macroscopic Dynamics of Spiking Neurons
Baldy, N., Breyton, M., Woodman, M., Jirsa, V., & Hashemi, M. (2024) · Neural Computation · doi:10.1162/neco_a_01701 · Cited by 10 · see citing works
Confidence and second-order errors in cortical circuits
Granier, A., Petrovici, M. A., Senn, W., & Wilmes, K. A. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae404 · Cited by 12 · see citing works
Gangopadhyay, A., & Chakrabartty, S. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.715451 · Cited by 5 · see citing works
Understanding and Improving Optimization in Predictive Coding Networks
Alonso, N., Krichmar, J. L., & Neftci, E. O. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i10.28954 · Cited by 5 · see citing works
Continual Sequence Modeling With Predictive Coding
Annabi, L., Pitti, A., & Quoy, M. (2022) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2022.845955 · Cited by 5 · see citing works
Preventing Deterioration of Classification Accuracy in Predictive Coding Networks
Kinghorn, P., Millidge, B., & Buckley, C. L. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-28719-0_1 · Cited by 4 · see citing works
Self-citationWhy Human Prejudice is so Persistent: A Predictive Coding Analysis
Hung, T.-W. (2023) · Social Epistemology · doi:10.1080/02691728.2023.2237942 · Cited by 4 · see citing works
Kappel, D., & Tetzlaff, C. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012531 · Cited by 6 · see citing works
Moldwin, T., Azran, L. S., & Segev, I. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012754 · Cited by 4 · see citing works
Banki-Koshki, H., & Seyyedsalehi, S. A. (2022) · Communications in Nonlinear Science and Numerical Simulation · doi:10.1016/j.cnsns.2022.107044 · Cited by 4 · see citing works
Fisher, A., & Rao, R. P. N. (2023) · PNAS Nexus · doi:10.1093/pnasnexus/pgad337 · Cited by 4 · see citing works
Tightening the Biological Constraints on Gradient-Based Predictive Coding
Alonso, N., & Neftci, E. O. (2021) · International Conference on Neuromorphic Systems (ICONS) · doi:10.1145/3477145.3477148 · Cited by 4 · see citing works
Variational Predictive Routing with Nested Subjective TimescalesPreprint
Zakharov, A., Guo, Q., & Fountas, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.11236 · Cited by 6 · see citing works
A Predictive Coding Model of the N400Preprint
Eddine, S. N., Brothers, T. A., Wang, L., Spratling, M., & Kuperberg, G. R. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.04.10.536279 · Cited by 4 · see citing works
FABLE: A Development and Computing Framework for Brain-inspired Learning Algorithms
Pang, M., Li, Y., Li, Z., & Zhang, Y. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10192026 · Cited by 3 · see citing works
Deep Hybrid Models: Infer and Plan in a Dynamic World
Priorelli, M., & Stoianov, I. P. (2025) · Entropy · doi:10.3390/e27060570 · Cited by 3 · see citing works
Learning distant cause and effect using only local and immediate credit assignment
Rawlinson, D., Ahmed, A., & Kowadlo, G. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533760 · Cited by 3 · see citing works
A synapse-centric account of the free energy principlePreprint
Kappel, D., & Tetzlaff, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.12649 · Cited by 3 · see citing works
Brain-like Flexible Visual Inference by Harnessing Feedback-Feedforward Alignment.Preprint
Toosi, T., & Issa, E. B. (2023) · PubMed · Cited by 3 · see citing works
Local Back-Propagation for Forward-Forward Networks: Independent Unsupervised Layer-Wise Training
Hwang, T., Seo, H., & Jung, S. (2025) · Applied Sciences · doi:10.3390/app15158207 · Cited by 2 · see citing works
Modeling natural neural networks of decision making with artificial neural networks
Funamizu, A., & Karakida, R. (2025) · Neuroscience Research · doi:10.1016/j.neures.2025.104961 · Cited by 2 · see citing works
Calibration variance outperforms predictive uncertainty in activity-only allostatic load
Tkemaladze, J. (2026) · Longevity Horizon · doi:10.65649/eqwt3660 · Cited by 2 · see citing works
PCGC: a performance compact graph compiler based on multilevel fusion-splitting rules
Dong, D., Jiang, H., Lin, H., & Song, Y. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-023-05298-w · Cited by 2 · see citing works
Considerations of Biological Plausibility in Deep Learning
Campbell, J. (2022) · Cornell Undergraduate Research Journal · doi:10.37513/curj.v1i1.660 · Cited by 2 · see citing works
Learning Sentence-Level Representations with Predictive Coding
Araujo, V., Moens, M., & Soto, Á. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5010005 · Cited by 3 · see citing works
Predictive coding feedback results in perceived illusory contours in a recurrent neural networkPreprint
Pang, Z., O’May, C. B., Choksi, B., & Rufin Vanrullen, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01955 · Cited by 2 · see citing works
Bilevel Programming and Deep Learning: A Unifying View on Inference Learning Methods.Preprint
Zach, C. (2021) · arXiv (Cornell University) · Cited by 2 · see citing works
Wahidi, M., Rizk, A., & Imad, R. (2025) · IEEE Access · doi:10.1109/access.2025.3589514 · Cited by 1 · see citing works
Matsumoto, T., Fujii, K., Murata, S., & Tani, J. (2025) · Entropy · doi:10.3390/e27080846 · Cited by 1 · see citing works
AJISAI: Augmented Joint Information Self Supervision with Aligned Inputs
Dewanto, B., Luthfiarta, A., Nugraha, A., Firdausillah, F., Fitri, M., & Marendratama, R. (2025) · International Seminar on Application for Technology of Information and Communication (iSemantic) · doi:10.1109/isemantic67418.2025.11291793 · Cited by 1 · see citing works
Imran, B., Riadi, S., Basri, H., Erniwati, S., Zaeniah, Muahidin, Z., Efendi, M. M., & Yani, A. (2025) · International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS) · doi:10.1109/icimcis68501.2025.11326933 · Cited by 1 · see citing works
Only strict saddles in the energy landscape of predictive coding networks? *
Innocenti, F., Achour, E. M., Singh, R. R., & Buckley, C. L. (2025) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/ade2eb · Cited by 1 · see citing works
Self-citation‘Backpropagation and the brain’ realized in cortical error neuron microcircuits
Max, K., Jaras, I., Granier, A., Wilmes, K. A., & Petrovici, M. A. (2026) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1014164 · Cited by 1 · see citing works
Investigating the Scalability and Biological Plausibility of the Activation Relaxation AlgorithmPreprint
Millidge, B., Tschantz, A., Seth, A. K., & Buckley, C. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06219 · Cited by 1 · see citing works
Self-citationPredictive coding, precision and natural gradientsPreprint
Ofner, A., Ratul, R. K., Ghosh, S., & Stober, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.06942 · Cited by 1 · see citing works
Synapses learn to utilize pre-synaptic noise for the prediction of postsynaptic dynamicsPreprint
Kappel, D., & Tetzlaff, C. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.04.22.489175 · Cited by 1 · see citing works
Predictive processing in neuroscience, computational modeling and psychology
Brucklacher, M., Lee, K., Moreni, G., Mejías, J. F., Bohté, S. M., & Pennartz, C. M. A. (2024) · Elsevier eBooks · doi:10.1016/b978-0-12-820480-1.00201-1 · Cited by 1 · see citing works
Local Back-Propagation: Layer-wise Unsupervised Learning in Forward-Forward AlgorithmsPreprint
Hwang, T., Seo, H., & Jung, S. (2025) · Research Square · doi:10.21203/rs.3.rs-5695830/v1 · Cited by 1 · see citing works
The generalized Hierarchical Gaussian Filter
Weber, L. A., Waade, P. T., Legrand, N., Møller, A. H., Stephan, K. E., & Mathys, C. (2026) · eLife · doi:10.7554/elife.110174
Closed-form predictive coding via hierarchical Gaussian filters
Baskakovs, A., Estebe, S., Enevoldsen, K., Nielbo, K., Mathys, C., & Legrand, N. (2026) · arXiv · doi:10.48550/arxiv.2605.20293
“The large majority of PCNs developed for machine-learning tasks [Whittington and Bogacz, 2017, Salvatori et al., 2021, 2024, Millidge et al., 2022, Pinchetti et al., 2025, Oliviers et al., 2025, Song et al., 2024] fix precisions to the identity throughout training, effectively assuming…”
InfluentialEffective methods and framework for energy-based local learning of deep neural networks
Chen, H., Yang, B., He, F., Zhou, F., Chen, S., Wu, C., Li, F., & Chua, Y. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1605706
Ma, G., Chen, Y., Zhang, Y., Guo, L., Wu, Y., & Yu, S. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.133139
The generalized Hierarchical Gaussian Filter
Weber, L. A., Waade, P. T., Legrand, N., Møller, A. H., Stephan, K. E., & Mathys, C. (2026) · doi:10.7554/elife.110174.1
Deep learning and neuroscience : a match made in heaven?
Choksi, B. (2022) · theses.fr (ABES) · doi:10.70675/42802d90z0c77z4de8z829dz508145498217
L, G., & Yu, S. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464514
Suzuki, K. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1819038
Canay, Ö. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6915201
Weight Feedback Computes the Jacobian Transpose Locally in Modern Deep Networks
Shen, J., & Li, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37013-6_32
Bio-Inspired Artificial Neural Networks Based on Predictive Coding [Lecture Notes]
Casnici, D., Frenkel, C., Dauwels, J., & Capobianco, R. (2026) · IEEE Signal Processing Magazine · doi:10.1109/msp.2026.3666766
The phantastic organ: predictive processing in the brain
Friston, K. J., & Parr, T. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1901293
Associative Memories via Predictive Coding
Salvatori, T., Song, Y., Hong, Y., Frieder, S., Sha, L., Xu, Z., Bogacz, R., & Lukasiewicz, T. (2021) · arXiv · doi:10.48550/arxiv.2109.08063 · Cited by 33 · see citing works
- Authors
- Millidge, B., Tschantz, A., & Buckley, C. L.
- Affiliations
- University of Edinburgh (GB)
- University of Sussex (GB)
- Corresponding author
- Millidge, Beren
- Tschantz, Alexander
- Buckley, Christopher L.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2022-05-19
- Volume
- 34
- Issue
- 6
- Pages
- 1329-1368
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W3033108376
- Cited by
- 95
- References
- 113
- Citations by year
- 2026: 11 · 2025: 15 · 2024: 11 · 2023: 16 · 2022: 19 · 2021: 19 · 2020: 4
- FWCI
- 5.02
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- University of Sussex
- Engineering and Physical Sciences Research Council
- Biotechnology and Biological Sciences Research Council (BB/P022197/1)
- National Institutes of Natural Sciences (01112005)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- predictive coding, backpropagation, automatic differentiation, computational graphs, Hebbian learning, local learning rules, gradient approximation, convolutional neural networks, recurrent neural networks, long short-term memory, neuromorphic architectures, neural circuits
- MeSH terms
- Machine Learning, Algorithms, Neurons, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
