A survey on neuro-mimetic deep learning via predictive coding
Open accessAbstract
Artificial intelligence (AI) is rapidly becoming one of the key technologies of this century. The majority of results in AI thus far have been achieved using deep neural networks trained with a learning algorithm called error backpropagation, always considered biologically implausible. To this end, recent works have studied learning algorithms for deep neural networks inspired by the neurosciences. One such theory, called predictive coding (PC), has shown promising properties that make it potentially valuable for the machine learning community: it can model information processing in different areas of the brain, can be used in control and robotics, has a solid mathematical foundation in variational inference, and performs its computations asynchronously. Inspired by such properties, works that propose novel PC-like algorithms are starting to be present in multiple sub-fields of machine learning and AI at large. Here, we survey such efforts by first providing a broad overview of the history of PC to provide common ground for the understanding of the recent developments, then by describing current efforts and results, and concluding with a large discussion of possible implications and ways forward.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(2014) · Cited by 33,419 · see citing works
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Hodgkin, A. L., & Huxley, A. F. (1952) · The Journal of Physiology · doi:10.1113/jphysiol.1952.sp004764 · Cited by 23,191 · see citing works
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,037 · see citing works
Strukov, D. B., Snider, G. S., Stewart, D. R., & Williams, R. S. (2008) · Nature · doi:10.1038/nature06932 · Cited by 11,744 · see citing works
Memristor—The Missing Circuit Element
Chua, L. (1971) · IEEE Transactions on Circuit Theory · doi:10.1109/tct.1971.1083337 · Cited by 10,268 · see citing works
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · see citing works
Self-citationindicated, N. A. (1950) · Journal of Consulting Psychology · doi:10.1037/h0049612 · Cited by 6,984 · see citing works
Whatever next? Predictive brains, situated agents, and the future of cognitive science
Clark, A. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12000477 · Cited by 5,924 · see citing works
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · see citing works
Self-citationBi, G.-Q., & Poo, M.-M. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-24-10464.1998 · Cited by 4,744 · see citing works
A theory of cortical responses.
Friston, K. (2005) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2005.1622 · Cited by 4,578 · see citing works
Self-citationA million spiking-neuron integrated circuit with a scalable communication network and interface
Merolla, P. A., Arthur, J. V., Alvarez-Icaza, R., Cassidy, A. S., Sawada, J., Akopyan, F., Jackson, B. L., Imam, N., Chen, G., Nakamura, Y., et al. (2014) · Science · doi:10.1126/science.1254642 · Cited by 4,137 · see citing works
Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999) · Statistical Science · doi:10.1214/ss/1009212519 · Cited by 4,118 · see citing works
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning
Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G. N., Cao, Y., Choday, S. H., Dimou, G. D., Joshi, P., Imam, N., Jain, S., et al. (2018) · IEEE Micro · doi:10.1109/mm.2018.112130359 · Cited by 3,846 · see citing works
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642 · see citing works
Networks of spiking neurons: The third generation of neural network models
Maass, W. (1997) · Neural Networks · doi:10.1016/s0893-6080(97)00011-7 · Cited by 3,551 · see citing works
Hierarchical Dirichlet Processes
Teh, Y. W., Jordan, M. I., Beal, M. J., & Blei, D. M. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000302 · Cited by 3,445 · see citing works
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.11487 · Cited by 2,528 · see citing works
Canonical Microcircuits for Predictive Coding
Bastos, A. M., Usrey, W. M., Adams, R. A., Mangun, G. R., Fries, P., & Friston, K. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.038 · Cited by 2,421 · see citing works
Self-citationInternational Conference on Artificial Neural Networks
Boubacar, H. A., & Lecoeuche, S. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90065-s · Cited by 2,088 · see citing works
How to grow a mind: Statistics, structure, and abstraction
Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman, N. D. (2011) · Science · doi:10.1126/science.1192788 · Cited by 1,815 · see citing works
Proceedings for the Annual Meeting of the Cognitive Science Society
Dziemianko, M., Clarke, A. D. F., & Keller, F. (2013) · Cited by 1,726 · see citing works
Predictive coding under the free-energy principle
Friston, K., & Kiebel, S. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0300 · Cited by 1,579 · see citing works
Self-citationHierarchical Bayesian inference in the visual cortex
Lee, T. S., & Mumford, D. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001434 · Cited by 1,473 · see citing works
Generative Modeling by Estimating Gradients of the Data Distribution
Song, Y., & Ermon, S. (2019) · arXiv · doi:10.48550/arxiv.1907.05600 · Cited by 1,453 · see citing works
The mismatch negativity: a review of underlying mechanisms
Garrido, M. I., Kilner, J. M., Stephan, K. E., & Friston, K. J. (2009) · Clinical Neurophysiology · doi:10.1016/j.clinph.2008.11.029 · Cited by 1,441 · see citing works
Self-citationFurber, S. B., Galluppi, F., Temple, S., & Plana, L. A. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2304638 · Cited by 1,373 · see citing works
Attention, uncertainty, and free-energy
Feldman, H., & Friston, K. J. (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00215 · Cited by 1,365 · see citing works
Self-citationMathematical Foundations of Neuroscience
Ermentrout, G. B., & Terman, D. H. (2010) · Interdisciplinary Applied Mathematics · doi:10.1007/978-0-387-87708-2 · Cited by 1,340 · see citing works
2012 IEEE Conference on Computer Vision and Pattern Recognition
(2012) · doi:10.1109/cvpr17489.2012 · Cited by 1,272 · see citing works
A perspective view and survey of meta-learning
Vilalta, R., & Drissi, Y. (2002) · Artificial Intelligence Review · doi:10.1023/a:1019956318069 · Cited by 1,108 · see citing works
Predictive coding: a fresh view of inhibition in the retina
Srinivasan, M. V., Laughlin, S. B., & Dubs, A. (1982) · Proceedings of the Royal Society of London. Series B. Biological Sciences · doi:10.1098/rspb.1982.0085 · Cited by 1,042 · see citing works
Grollier, J., Querlioz, D., Camsari, K. Y., Everschor‐Sitte, K., Fukami, S., & Stiles, M. D. (2020) · Nature Electronics · doi:10.1038/s41928-019-0360-9 · Cited by 990 · see citing works
Bayesian Model Selection and Model Averaging
Wasserman, L. (2000) · Journal of Mathematical Psychology · doi:10.1006/jmps.1999.1278 · Cited by 989 · see citing works
Kanerva, P. (2009) · Cognitive Computation · doi:10.1007/s12559-009-9009-8 · Cited by 977 · see citing works
Kanerva, P. (1988) · Medical Entomology and Zoology · Cited by 967 · see citing works
Hierarchical Models in the Brain
Friston, K. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000211 · Cited by 952 · see citing works
Self-citationCommunication dynamics in complex brain networks
Avena-Koenigsberger, A., Misic, B., & Sporns, O. (2018) · Nature Reviews Neuroscience · doi:10.1038/nrn.2017.149 · Cited by 856 · see citing works
Ten simple rules for dynamic causal modeling
Stephan, K. E., Penny, W. D., Moran, R. J., den Ouden, H. E. M., Daunizeau, J., & Friston, K. J. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.11.015 · Cited by 803 · see citing works
Self-citationThe electrical constants of a crustacean nerve fibre
Hodgkin, A. L., & Rushton, W. A. H. (1946) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.1946.0024 · Cited by 773 · see citing works
Learning and inference in the brain
Friston, K. (2003) · Neural Networks · doi:10.1016/j.neunet.2003.06.005 · Cited by 724 · see citing works
Self-citation2014 International Joint Conference on Neural Networks (IJCNN)
(2014) · doi:10.1109/ijcnn32435.2014 · Cited by 699 · see citing works
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., O⿿Doherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674 · see citing works
Self-citationAction understanding and active inference
Friston, K., Mattout, J., & Kilner, J. (2011) · Biological Cybernetics · doi:10.1007/s00422-011-0424-z · Cited by 665 · see citing works
Self-citation(2005) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.5555/1046920.1088695 · Cited by 650 · see citing works
2019 International Joint Conference on Neural Networks (IJCNN)
(2019) · doi:10.1109/ijcnn46175.2019 · Cited by 646 · see citing works
Correlation between neural spike trains increases with firing rate
de la Rocha, J., Doiron, B., Shea-Brown, E., Josić, K., & Reyes, A. (2007) · Nature · doi:10.1038/nature06028 · Cited by 644 · see citing works
Theory of Physiological Properties of Dendrites
Rall, W. (1962) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1962.tb54120.x · Cited by 609 · see citing works
A tutorial on Bayesian nonparametric models
Gershman, S. J., & Blei, D. M. (2011) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2011.08.004 · Cited by 606 · see citing works
Dynamical memristors for higher-complexity neuromorphic computing
Kumar, S., Wang, X., Strachan, J. P., Yang, Y., & Lu, W. D. (2022) · Nature Reviews Materials · doi:10.1038/s41578-022-00434-z · Cited by 573 · see citing works
Predictive coding explains binocular rivalry: an epistemological review
Hohwy, J., Roepstorff, A., & Friston, K. (2008) · Cognition · doi:10.1016/j.cognition.2008.05.010 · Cited by 546 · see citing works
Self-citationA neuronal model of predictive coding accounting for the mismatch negativity.
Wacongne, C., Changeux, J.-P., & Dehaene, S. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5003-11.2012 · Cited by 542 · see citing works
Equilibrium Propagation: Bridging the Gap between Energy-Based Models and Backpropagation
Scellier, B., & Bengio, Y. (2017) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2017.00024 · Cited by 504 · see citing works
Active inference, sensory attenuation and illusions
Brown, H., Adams, R. A., Parees, I., Edwards, M., & Friston, K. (2013) · Cognitive Processing · doi:10.1007/s10339-013-0571-3 · Cited by 488 · see citing works
Self-citationPerceptions as Hypotheses: Saccades as Experiments
Friston, K., Adams, R. A., Perrinet, L., & Breakspear, M. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00151 · Cited by 462 · see citing works
Self-citationThe Role of Occam's Razor in Knowledge Discovery
Domingos, P. (1999) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009868929893 · Cited by 459 · see citing works
A review of predictive coding algorithms
Spratling, M. W. (2017) · Brain and Cognition · doi:10.1016/j.bandc.2015.11.003 · Cited by 437 · see citing works
Eliaš, P. (1955) · IEEE Transactions on Information Theory · doi:10.1109/tit.1955.1055126 · Cited by 436 · see citing works
Cerebral hierarchies: predictive processing, precision and the pulvinar
Kanai, R., Komura, Y., Shipp, S., & Friston, K. (2015) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2014.0169 · Cited by 432 · see citing works
Self-citationGreat Expectations: Is there Evidence for Predictive Coding in Auditory Cortex?
Heilbron, M., & Chait, M. (2018) · Neuroscience · doi:10.1016/j.neuroscience.2017.07.061 · Cited by 428 · see citing works
Theories of Error Back-Propagation in the Brain
Whittington, J. C. R., & Bogacz, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.005 · Cited by 420 · see citing works
The graphical brain: Belief propagation and active inference
Friston, K. J., Parr, T., & de Vries, B. (2017) · Network Neuroscience · doi:10.1162/netn_a_00018 · Cited by 415 · see citing works
Self-citationLearning Complex, Extended Sequences Using the Principle of History Compression
Schmidhuber, J. (1992) · Neural Computation · doi:10.1162/neco.1992.4.2.234 · Cited by 398 · see citing works
Laird, J. E., Lebiere, C., & Rosenbloom, P. S. (2017) · AI Magazine · doi:10.1609/aimag.v38i4.2744 · Cited by 392 · see citing works
Probabilistic Models of the Brain : Perception and Neural Function
Rao, R. P. N., Olshausen, B. A., & Lewicki, M. S. (2002) · Cited by 385 · see citing works
Self-citationThe history of the future of the Bayesian brain
Friston, K. J. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.10.004 · Cited by 346 · see citing works
Self-citationNeural elements for predictive coding
Shipp, S. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01792 · Cited by 340 · see citing works
Catalyzing next-generation Artificial Intelligence through NeuroAI
Zador, A. M., Escola, G. S., Richards, B. A., Ölveczky, B. P., Bengio, Y., Boahen, K. A., Botvinick, M., Chklovskii, D. B., Churchland, A. K., Claudia Clopath, A., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-37180-x · Cited by 337 · see citing works
Evaluating the neurophysiological evidence for predictive processing as a model of perception
Walsh, K. S., McGovern, D. P., Clark, A., & O'Connell, R. G. (2020) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.14321 · Cited by 333 · see citing works
Visual Feature Extraction by a Multilayered Network of Analog Threshold Elements
Fukushima, K. (1969) · IEEE Transactions on Systems Science and Cybernetics · doi:10.1109/tssc.1969.300225 · Cited by 323 · see citing works
2024 IEEE International Conference on Robotics and Automation (ICRA)
Mortimer, P., Hagmanns, R., Granero, M., Luettel, T., Petereit, J., & Wuensche, H.-J. (2024) · doi:10.1109/icra57147.2024 · Cited by 314 · see citing works
Free Energy, Precision and Learning: The Role of Cholinergic Neuromodulation
Moran, R. J., Campo, P., Symmonds, M., Stephan, K. E., Dolan, R. J., & Friston, K. J. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.4255-12.2013 · Cited by 312 · see citing works
Self-citationEvidence of a predictive coding hierarchy in the human brain listening to speech
Caucheteux, C., Gramfort, A., & King, J.-R. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01516-2 · Cited by 308 · see citing works
Repetition suppression and its contextual determinants in predictive coding
Auksztulewicz, R., & Friston, K. (2016) · Cortex · doi:10.1016/j.cortex.2015.11.024 · Cited by 307 · see citing works
Self-citationComparing dynamic causal models using AIC, BIC and free energy
Penny, W. D. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.07.039 · Cited by 298 · see citing works
Organoid intelligence (OI): the new frontier in biocomputing and intelligence-in-a-dish
Smirnova, L., Caffo, B. S., Gracias, D. H., Huang, Q., Pantoja, I. E. M., Tang, B., Zack, D. J., Berlinicke, C. A., Boyd, J. L., Harris, T. D., et al. (2023) · Frontiers in Science · doi:10.3389/fsci.2023.1017235 · Cited by 290 · see citing works
Dynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex
Rao, R. P. N., & Ballard, D. H. (1997) · Neural Computation · doi:10.1162/neco.1997.9.4.721 · Cited by 290 · see citing works
Self-citationA tutorial on the free-energy framework for modelling perception and learning
Bogacz, R. (2017) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.11.003 · Cited by 283 · see citing works
DEM: A variational treatment of dynamic systems
Friston, K. J., Trujillo-Barreto, N., & Daunizeau, J. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.02.054 · Cited by 278 · see citing works
Self-citationWhittington, J. C. R., & Bogacz, R. (2017) · Neural Computation · doi:10.1162/neco_a_00949 · Cited by 278 · see citing works
Minimum Error Entropy Kalman Filter
Chen, B., Dang, L., Gu, Y., Zheng, N., & Príncipe, J. C. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2957269 · Cited by 276 · see citing works
Layer and rhythm specificity for predictive routing
Bastos, A. M., Lundqvist, M., Waite, A. S., Kopell, N., & Miller, E. K. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2014868117 · Cited by 268 · see citing works
Post hoc Bayesian model selection
Friston, K. J., & Penny, W. D. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.03.062 · Cited by 260 · see citing works
Self-citationMeta Fundamental AI Research Diplomacy Team (FAIR)†, Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., et al. (2022) · Science · doi:10.1126/science.ade9097 · Cited by 241 · see citing works
Reflections on agranular architecture: predictive coding in the motor cortex
Shipp, S., Adams, R. A., & Friston, K. J. (2013) · Trends in Neurosciences · doi:10.1016/j.tins.2013.09.004 · Cited by 239 · see citing works
Self-citationPrediction and memory: A predictive coding account
Barron, H. C., Auksztulewicz, R., & Friston, K. (2020) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2020.101821 · Cited by 220 · see citing works
Self-citationPredictive coding as a model of response properties in cortical area V1
Spratling, M. W. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4911-09.2010 · Cited by 206 · see citing works
2019 IEEE International Solid-State Circuits Conference - (ISSCC)
(2019) · doi:10.1109/isscc19946.2019 · Cited by 192 · see citing works
Parr, T., & Friston, K. J. (2018) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2018.10.006 · Cited by 174 · see citing works
Self-citationFriston, K., Stephan, K., Li, B., & Daunizeau, J. (2010) · Mathematical Problems in Engineering · doi:10.1155/2010/621670 · Cited by 171 · see citing works
Self-citationAn optimal estimation approach to visual perception and learning
Rao, R. P. N. (1999) · Vision Research · doi:10.1016/s0042-6989(98)00279-x · Cited by 164 · see citing works
Self-citationMastering the game of Stratego with model-free multiagent reinforcement learning
Pérolat, J., Vylder, B. D., Hennes, D., Tarassov, E., Strub, F., de Boer, V., Müller, P. F., Connor, J. T., BURCH, N. R., Anthony, T., et al. (2022) · Science · doi:10.1126/science.add4679 · Cited by 163 · see citing works
Where is the error? Hierarchical predictive coding through dendritic error computation
Mikulasch, F. A., Rudelt, L., Wibral, M., & Priesemann, V. (2022) · Trends in Neurosciences · doi:10.1016/j.tins.2022.09.007 · Cited by 128 · see citing works
Computational Phenotyping in Psychiatry: A Worked Example
Schwartenbeck, P., & Friston, K. J. (2016) · eNeuro · doi:10.1523/eneuro.0049-16.2016 · Cited by 124 · see citing works
Self-citationA predictive coding account of bistable perception - a model-based fMRI study
Weilnhammer, V., Stuke, H., Hesselmann, G., Sterzer, P., & Schmack, K. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005536 · Cited by 120 · see citing works
Kuchling, F., Friston, K. J., Georgiev, G., & Levin, M. G. (2019) · Physics of Life Reviews · doi:10.1016/j.plrev.2019.06.001 · Cited by 118 · see citing works
Self-citationAn Active Inference Approach to Modeling Structure Learning: Concept Learning as an Example Case
Smith, R. S., Schwartenbeck, P., Parr, T., & Friston, K. J. (2020) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2020.00041 · Cited by 111 · see citing works
Self-citationPerception and hierarchical dynamics
Kiebel, S. J. (2009) · Frontiers in Neuroinformatics · doi:10.3389/neuro.11.020.2009 · Cited by 106 · see citing works
Toward the neural implementation of structure learning
Tervo, D. G. R., Tenenbaum, J. B., & Gershman, S. J. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.01.014 · Cited by 106 · see citing works
Illusory Motion Reproduced by Deep Neural Networks Trained for Prediction
Watanabe, E., Kitaoka, A., Sakamoto, K., Yasugi, M., & Tanaka, K. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00345 · Cited by 101 · see citing works
Is predictive coding theory articulated enough to be testable?
Kogo, N., & Trengove, C. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00111 · Cited by 98 · see citing works
Brain computation by assemblies of neurons
Papadimitriou, C. H., Vempala, S., Mitropolsky, D., Collins, M., & Maass, W. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2001893117 · Cited by 87 · see citing works
Inferring neural activity before plasticity as a foundation for learning beyond backpropagation
Song, Y., Millidge, B., Salvatori, T., Lukasiewicz, T., Xu, Z., & Bogacz, R. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01514-1 · Cited by 85 · see citing works
Self-citationPredictive coding, precision and synchrony
Friston, K. J. (2012) · Cognitive Neuroscience · doi:10.1080/17588928.2012.691277 · Cited by 82 · see citing works
Self-citationFicco, L., Mancuso, L., Manuello, J., Teneggi, A., Liloia, D., Duca, S., Costa, T., Kovács, G., & Cauda, F. (2021) · Scientific Reports · doi:10.1038/s41598-021-95603-5 · Cited by 77 · see citing works
Proceedings of the 2019 on International Conference on Multimedia Retrieval
El Saddik, A., Canada, I. '. I. C. O. M. R. 2. O. O., Association for Computing Machinery-Digital Library, ACM Special Interest Group on Multimedia Systems, & SIGs, A. (2019) · doi:10.1145/3323873 · Cited by 76 · see citing works
Somato-dendritic synaptic plasticity and error-backpropagation in active dendrites
Schiess, M., Urbanczik, R., & Senn, W. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004638 · Cited by 75 · see citing works
Gordon, N., Koenig‐Robert, R., Tsuchiya, N., van Boxtel, J. J. A., & Hohwy, J. (2017) · eLife · doi:10.7554/elife.22749 · Cited by 72 · see citing works
Active inference, eye movements and oculomotor delays
Perrinet, L., Adams, R. A., & Friston, K. J. (2014) · Biological Cybernetics · doi:10.1007/s00422-014-0620-8 · Cited by 69 · see citing works
Self-citationFree Energy and Dendritic Self-Organization
Kiebel, S. J., & Friston, K. J. (2011) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2011.00080 · Cited by 68 · see citing works
Self-citationA Hierarchical Predictive Coding Model of Object Recognition in Natural Images
Spratling, M. (2016) · Cognitive Computation · doi:10.1007/s12559-016-9445-1 · Cited by 65 · see citing works
An Empirical Study of Active Inference on a Humanoid Robot
Oliver, G., Lanillos, P., & Cheng, G. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3049907 · Cited by 65 · see citing works
Breeding Brains? Patients’ and Laymen’s Perspectives on Cerebral Organoids
Haselager, D. R., Boers, S. N., Jongsma, K. R., Vinkers, C. H., Broekman, M. L. D., & Bredenoord, A. L. (2020) · Regenerative Medicine · doi:10.2217/rme-2020-0108 · Cited by 61 · see citing works
Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representations
Ororbia, A. G., Mali, A., Giles, C. L., & Kifer, D. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2953622 · Cited by 60 · see citing works
Self-citationThe neural coding framework for learning generative models
Ororbia, A. G., & Kifer, D. (2022) · Nature Communications · doi:10.1038/s41467-022-29632-7 · Cited by 58 · see citing works
Self-citationPredictive Coding: A more cognitive process than we thought?
Gabhart, K. M., Xiong, Y. (., & Bastos, A. M. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2025.01.012 · Cited by 54 · see citing works
Hybrid predictive coding: Inferring, fast and slow
Tscshantz, A., Millidge, B., Seth, A. K., & Buckley, C. L. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011280 · Cited by 51 · see citing works
Self-citationDeep Predictive Coding Network with Local Recurrent Processing for Object RecognitionPreprint
Han, K., Wen, H., Zhang, Y., Fu, D., Culurciello, E., & Liu, Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07526 · Cited by 46 · see citing works
How Active Inference Could Help Revolutionise Robotics
Costa, L. D., Lanillos, P., Sajid, N., Friston, K. J., & Khan, S. (2022) · Entropy · doi:10.3390/e24030361 · Cited by 46 · see citing works
Self-citationBayesian Filtering with Multiple Internal Models: Toward a Theory of Social Intelligence
Isomura, T., Parr, T., & Friston, K. J. (2019) · Neural Computation · doi:10.1162/neco_a_01239 · Cited by 41 · see citing works
Self-citationSpiking neural predictive coding for continually learning from data streams
Ororbia, A. G. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126292 · Cited by 38 · see citing works
Self-citation2006 IEEE International Symposium on Information Theory
(2006) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2006.261826 · Cited by 36 · see citing works
Scalable and Practical Natural Gradient for Large-Scale Deep Learning
Osawa, K., Tsuji, Y., Ueno, Y., Naruse, A., Foo, C.-S., & Yokota, R. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3004354 · Cited by 35 · see citing works
Recurrent predictive coding models for associative memory employing covariance learning
Tang, M., Salvatori, T., Millidge, B., Song, Y., Lukasiewicz, T., & Bogacz, R. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010719 · Cited by 31 · see citing works
Self-citationEntropy coding in speech and television differential PCM systems (Corresp.)
O'Neal, J. B. (1971) · IEEE Transactions on Information Theory · doi:10.1109/tit.1971.1054706 · Cited by 30 · see citing works
Learning on Arbitrary Graph Topologies via Predictive Coding
Salvatori, T., Pinchetti, L., Millidge, B., Song, Y., Bao, T.-Y., Bogacz, R., & Lukasiewicz, T. (2022) · PubMed · doi:10.48550/arxiv.2201.13180 · Cited by 27 · see citing works
Self-citationPredify: Augmenting deep neural networks with brain-inspired predictive coding dynamicsPreprint
Choksi, B., Mozafari, M., O’May, C. B., Ador, B., Alamia, A., & Rufin Vanrullen, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02749 · Cited by 26 · see citing works
Deep kinematic inference affords efficient and scalable control of bodily movements
Priorelli, M., Pezzulo, G., & Stoianov, I. P. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2309058120 · Cited by 24 · see citing works
Fitting predictive coding to the neurophysiological data
Spratling, M. W. (2019) · Brain Research · doi:10.1016/j.brainres.2019.146313 · Cited by 23 · see citing works
Dynamic Expectation Maximization Algorithm for Estimation of Linear Systems with Colored Noise
Meera, A. A., & Wisse, M. (2021) · Entropy · doi:10.3390/e23101306 · Cited by 22 · see citing works
Wang, P., Goertzel, B., & Franklin, S. (2008) · Cited by 20 · see citing works
On the relationship between predictive coding and backpropagation
Rosenbaum, R. (2022) · PLoS ONE · doi:10.1371/journal.pone.0266102 · Cited by 20 · see citing works
Zucchet, N., & Sacramento, J. (2022) · Neural Computation · doi:10.1162/neco_a_01547 · Cited by 20 · see citing works
International Conference on Neuromorphic Systems 2020
(2020) · doi:10.1145/3407197 · Cited by 17 · see citing works
Lifelong Neural Predictive Coding: Learning Cumulatively Online Without Forgetting
Ororbia, A. G., Mali, A., Giles, C. L., & Kifer, D. (2022) · neural information processing systems · doi:10.52202/068431-0425 · Cited by 16 · see citing works
Self-citationCognitive Architectures for Sensory Processing
Príncipe, J. C., & Chalasani, R. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2307023 · Cited by 15 · see citing works
A neural model of binocular saccade planning and vergence control
Muhammad, W., & Spratling, M. (2015) · Adaptive Behavior · doi:10.1177/1059712315607363 · Cited by 14 · see citing works
Neuronal cultures playing Pong: First steps toward advanced screening and biological computing
Smirnova, L., & Härtung, T. (2022) · Neuron · doi:10.1016/j.neuron.2022.11.010 · Cited by 13 · see citing works
Structure Learning in Predictive Processing Needs Revision
Rutar, D., de Wolff, E., van Rooij, I., & Kwisthout, J. (2022) · Computational Brain & Behavior · doi:10.1007/s42113-022-00131-8 · Cited by 11 · see citing works
Gklezakos, D. C., & Rao, R. P. N. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.01.20.477125 · Cited by 10 · see citing works
Self-citationPredictive Coding as a Neuromorphic Alternative to Backpropagation: A Critical Evaluation
Zahid, U., Guo, Q., & Fountas, Z. (2023) · Neural Computation · doi:10.1162/neco_a_01620 · Cited by 10 · see citing works
Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural ImagesPreprint
Ororbia, A. G., & Mali, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.12047 · Cited by 10 · see citing works
Self-citationContext Dependent Encoding Using Convolutional Dynamic Networks
Chalasani, R., & Príncipe, J. C. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2360060 · Cited by 9 · see citing works
A Theoretical Framework for Inference Learning
Alonso, N., Millidge, B., Krichmar, J., & Neftci, E. O. (2022) · neural information processing systems · doi:10.52202/068431-2706 · Cited by 9 · see citing works
A single-spike model of predictive coding
Ballard, D. H., Rao, R. P. N., & Zhang, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00139-9 · Cited by 7 · see citing works
Self-citationBrain-like learning with exponentiated gradientsPreprint
Cornford, J. H., Pogodin, R., Ghosh, A., Sheng, K., Bicknell, B. A., Codol, O., Clark, B. A., Lajoie, G., & Richards, B. A. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.10.25.620272 · Cited by 7 · see citing works
Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representations
Sledge, I. J., & Príncipe, J. C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3115698 · Cited by 6 · see citing works
Learning probability distributions of sensory inputs with Monte Carlo predictive coding
Oliviers, G., Bogacz, R., & Meulemans, A. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012532 · Cited by 6 · see citing works
Poisson Variational Autoencoder
Vafaii, H., Galor, D., & Yates, J. L. (2024) · neural information processing systems · doi:10.52202/079017-1426 · Cited by 6 · see citing works
Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations
Araujo, V., Villa, A., Mendoza, M., Moens, M., & Soto, Á. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.240 · Cited by 4 · see citing works
BayesPCN: A Continually Learnable Predictive Coding Associative Memory
Yoo, J., & Wood, F. (2022) · neural information processing systems · doi:10.52202/068431-2168 · Cited by 4 · see citing works
(1952) · Victorian reports · doi:10.25291/vr/1952-vlr-361 · Cited by 3 · see citing works
Contrastive-Signal-Dependent Plasticity: Forward-Forward Learning of Spiking Neural SystemsPreprint
Ororbia, A. G. (2023) · doi:10.31219/osf.io/w2ztm · Cited by 2 · see citing works
Self-citationOn the Relationship Between Variational Inference and Auto-Associative MemoryPreprint
Annabi, L., Pitti, A., & Quoy, M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.08013 · Cited by 2 · see citing works
A Model of Predictive Coding based on Spike Timing
Ballard, D. H., Rao, R. P. N., & Zhang, Z. (1999) · Cited by 1 · see citing works
Self-citationContinual Competitive MemoryPreprint
Ororbia, A. G. (2021) · doi:10.31219/osf.io/yw6ua · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
2 of the 163 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
On Efficient Training of Large-Scale Deep Learning Models
Shen, L., Sun, Y., Yu, Z., Ding, L., Tian, X., & Tao, D. (2024) · ACM Computing Surveys · doi:10.1145/3700439 · Cited by 126 · see citing works
Mortal Computation: A Foundation for Biomimetic IntelligencePreprint
Ororbia, A. G., & Friston, K. J. (2023) · doi:10.31219/osf.io/epqkg · Cited by 33 · see citing works
Self-citationA Review of Neuroscience-Inspired Machine LearningPreprint
Ororbia, A. G., Mali, A., Kohan, A., Millidge, B., & Salvatori, T. (2024) · doi:10.31219/osf.io/uz8cv · Cited by 19 · see citing works
Self-citationContrastive signal–dependent plasticity: Self-supervised learning in spiking neural circuits
Ororbia, A. G. (2024) · Science Advances · doi:10.1126/sciadv.adn6076 · Cited by 18 · see citing works
Self-citationExploiting neuro-inspired dynamic sparsity for energy-efficient intelligent perception
Zhou, S., Gao, C., Delbrück, T., Verhelst, M., & Liu, S. (2025) · Nature Communications · doi:10.1038/s41467-025-65387-7 · Cited by 10 · see citing works
A spatiotemporal style transfer algorithm for dynamic visual stimulus generation
Greco, A., & Siegel, M. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00746-w · Cited by 10 · see citing works
Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit AssignmentPreprint
Ororbia, A. G. (2023) · doi:10.31219/osf.io/xukqf · Cited by 10 · see citing works
Self-citationDynamic planning in hierarchical active inference
Priorelli, M., & Stoianov, I. P. (2025) · Neural Networks · doi:10.1016/j.neunet.2024.107075 · Cited by 9 · see citing works
Predictive coding with spiking neural networks: A survey
N’dri, A. W., Gebhardt, W., Teulière, C., Zeldenrust, F., Rao, R. P. N., Triesch, J., & Ororbia, A. G. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108371 · Cited by 5 · see citing works
Self-citationOrorbia, A. G., & Kelly, M. A. (2023) · doi:10.31219/osf.io/z7c98 · Cited by 4 · see citing works
Self-citationPredictive Coding Networks and Inference Learning: Tutorial and Survey
van Zwol, B. H., Jefferson, R., & van den Broek, E. L. (2026) · ACM Computing Surveys · doi:10.1145/3797870 · Cited by 3 · see citing works
Predictive coding algorithms induce brain-like responses in artificial neural networks
Gütlin, D., & Auksztulewicz, R. (2025) · PLOS complex systems. · doi:10.1371/journal.pcsy.0000076 · Cited by 3 · see citing works
The sixth sense of the brain: interoception in organoid study
Li, J., Wang, F., Gao, Q., Chen, X., Bai, L., & Su, J. (2026) · Bone Research · doi:10.1038/s41413-026-00569-7 · Cited by 1 · see citing works
Beyond Syntax: How Do LLMs Understand Code?
North, M., Atapour–Abarghouei, A., & Bencomo, N. (2025) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier66352.2025.00023 · Cited by 1 · see citing works
Wahidi, M., Rizk, A., & Imad, R. (2025) · IEEE Access · doi:10.1109/access.2025.3589514 · Cited by 1 · see citing works
Candia‐Rivera, D. (2026) · Physics of Life Reviews · doi:10.1016/j.plrev.2026.06.003 · Cited by 1 · see citing works
Active sensing with predictive coding and uncertainty minimization
Sharafeldin, A., Imam, N., & Choi, H. (2024) · Patterns · doi:10.1016/j.patter.2024.100983 · Cited by 1 · see citing works
Closed-form predictive coding via hierarchical Gaussian filters
Baskakovs, A., Estebe, S., Enevoldsen, K., Nielbo, K., Mathys, C., & Legrand, N. (2026) · arXiv · doi:10.48550/arxiv.2605.20293
“…1999, Friston, 2005, Friston and Kiebel, 2009] offers a biologically grounded alternative in which each layer carries explicit prediction-error units, inference and learning both proceed by local energy minimisation, and weight updates are Hebbian [Salvatori et al., 2026, van Zwol et al., 2026].”
InfluentialOrorbia, A. G., & Kelly, M. A. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v2i1.27702
Self-citationEfficient Online Learning with Predictive Coding Networks: Exploiting Temporal CorrelationsPreprint
Zadeh-Jousdani, D. M., Hajizada, E., & Hüllermeier, E. (2025) · doi:10.22541/au.176463784.45436509/v1
Beni, M. D. (2026) · Journal for General Philosophy of Science · doi:10.1007/s10838-025-09759-z
Koudahl, M., Tschantz, A., Salvatori, T., Linander, H., Costa, L. D., Beck, J. M., & Buckley, C. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-16955-6_4
Self-citationBio-inspired backpropagation-free training for optical neural networks
Li, T., Dong, Y., Tu, K., Zhang, Y., Li, S., Duan, Y., Sun, P., Gu, M., & Wang, J. (2026) · Light Science & Applications · doi:10.1038/s41377-026-02394-3
Nothing beyond randomness is required for forward-forward learning
Zhou, Y., Hu, X., & Duan, S. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.115214
- Authors
- Salvatori, T., Mali, A., Buckley, C. L., Lukasiewicz, T., Rao, R. P. N., Friston, K., & Ororbia, A.
- Affiliations
- TU Wien (AT)
- University of South Florida (US)
- University of Sussex (GB)
- University of Washington (US)
- Wellcome Centre for Human Neuroimaging (GB)
- Rochester Institute of Technology (US)
- Corresponding author
- Salvatori, Tommaso
- ORCID
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2026-03
- Volume
- 195
- Pages
- 108161
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W4415019933
- Cited by
- 2
- References
- 163
- Citations by year
- 2026: 2
- FWCI
- 0.19
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Neural Networks (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- The Alan Turing Institute
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- predictive coding, backpropagation, variational inference, asynchronous computation
- MeSH terms
- Machine Learning, Deep Learning, Algorithms, Artificial Intelligence, Humans, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
