Hybrid predictive coding: Inferring, fast and slow
Open accessThe DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
Predictive coding is an influential model of cortical neural activity. It proposes that perceptual beliefs are furnished by sequentially minimising “prediction errors”—the differences between predicted and observed data. Implicit in this proposal is the idea that successful perception requires multiple cycles of neural activity. This is at odds with evidence that several aspects of visual perception—including complex forms of object recognition—arise from an initial “feedforward sweep” that occurs on fast timescales which preclude substantial recurrent activity. Here, we propose that the feedforward sweep can be understood as performing amortized inference (applying a learned function that maps directly from data to beliefs) and recurrent processing can be understood as performing iterative inference (sequentially updating neural activity in order to improve the accuracy of beliefs). We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner by describing both in terms of a dual optimization of a single objective function. We show that the resulting scheme can be implemented in a biologically plausible neural architecture that approximates Bayesian inference utilising local Hebbian update rules. We demonstrate that our hybrid predictive coding model combines the benefits of both amortized and iterative inference—obtaining rapid and computationally cheap perceptual inference for familiar data while maintaining the context-sensitivity, precision, and sample efficiency of iterative inference schemes. Moreover, we show how our model is inherently sensitive to its uncertainty and adaptively balances iterative and amortized inference to obtain accurate beliefs using minimum computational expense. Hybrid predictive coding offers a new perspective on the functional relevance of the feedforward and recurrent activity observed during visual perception and offers novel insights into distinct aspects of visual phenomenology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Whatever next? Predictive brains, situated agents, and the future of cognitive science
Clark, A. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12000477 · Cited by 5,924
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 2×
“In addition, given that predictive coding has been proposed as biological process theory of perception [8, 41, 42, 64], and as a way to explain the phenomenology of perceptual experience in terms of neural mechanisms [65, 66], our novel architecture also offers insights into why gist perception and focal perception ha…”
BackgroundMethodProbability Theory: The Logic of Science
Jaynes, E. T. (2003) · doi:10.1017/cbo9780511790423 · Cited by 4,045
Speed of processing in the human visual system
Thorpe, S., Fize, D., & Marlot, C. (1996) · Nature · doi:10.1038/381520a0 · Cited by 3,538
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588 · cited 1×
“Such considerations have motivated the popular view that the brain uses a version of Bayesian inference [7, 68], which describes the process of forming probabilistic beliefs about the causes of data, to accomplish perception.”
BackgroundCanonical Microcircuits for Predictive Coding
Bastos, A. M., Usrey, W. M., Adams, R. A., Mangun, G. R., Fries, P., & Friston, K. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.038 · Cited by 2,421
The distinct modes of vision offered by feedforward and recurrent processing
Lamme, V. A. F., & Roelfsema, P. R. (2000) · Trends in Neurosciences · doi:10.1016/s0166-2236(00)01657-x · Cited by 2,338 · cited 3×
“These and other disparate findings have fueled a long-standing debate over the respective contributions of bottom-up and top-down signals to visual perceptual content [9, 18–21].”
BackgroundResultA View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187
Hohwy, J. (2013) · doi:10.1093/acprof:oso/9780199682737.001.0001 · Cited by 1,914
How does the brain solve visual object recognition?
DiCarlo, J. J., Zoccolan, D., & Rust, N. C. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.010 · Cited by 1,790
Predictive coding under the free-energy principle
Friston, K., & Kiebel, S. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0300 · Cited by 1,579
Top-down versus bottom-up attentional control: a failed theoretical dichotomy
Awh, E., Belopolsky, A. V., & Theeuwes, J. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.06.010 · Cited by 1,557
Hierarchical Bayesian inference in the visual cortex
Lee, T. S., & Mumford, D. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001434 · Cited by 1,473
Top–down and bottom–up control of visual selection
Theeuwes, J. (2010) · Acta Psychologica · doi:10.1016/j.actpsy.2010.02.006 · Cited by 1,385 · cited 1×
“In visual perception, top-down signalling has long been recognised as playing several important functional roles [9]—for instance, in attentional modulation [10], in the goaldirected shaping of stimulus selection [11, 12], and in establishing recurrent loops that have been associated with conscious experience [13].”
BackgroundContinual Learning with Deep Generative Replay
Shin, H., Lee, J. K., Kim, J., & Kim, J. (2017) · arXiv · doi:10.48550/arxiv.1705.08690 · Cited by 1,385
A free energy principle for the brain
Friston, K., Kilner, J., & Harrison, L. (2006) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2006.10.001 · Cited by 1,369
Visual areas exert feedforward and feedback influences through distinct frequency channels
Bastos, A. M., Vezoli, J., Bosman, C. A., Schoffelen, J.-M., Oostenveld, R., Dowdall, J. R., De Weerd, P., Kennedy, H., & Fries, P. (2015) · Neuron · doi:10.1016/j.neuron.2014.12.018 · Cited by 1,320
Keeping the neural networks simple by minimizing the description length of the weights
Hinton, G. E., & van Camp, D. (1993) · Proceedings of the sixth annual conference on Computational learning theory - COLT '93 · doi:10.1145/168304.168306 · Cited by 994 · cited 1×
“Many works in machine learning have considered the notion of iterative and amortized (variational) inference [40, 51, 56], with recent work combining iterative and amortized inference in the context of perception [46] and control [44, 45, 57, 58].”
BackgroundHierarchical organization and functional streams in the visual cortex
Van Essen, D. C., & Maunsell, J. H. R. (1983) · Trends in Neurosciences · doi:10.1016/0166-2236(83)90167-4 · Cited by 962
Hierarchical Models in the Brain
Friston, K. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000211 · Cited by 952 · cited 2×
“At the same time, the predictive coding architecture has gained prominence as a process theory which could provide a neurobiological implementation of approximate Bayesian inference [8, 31, 37, 42, 84].”
BackgroundA cortical mechanism for triggering top-down facilitation in visual object recognition
Bar, M. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903321662976 · Cited by 900
Seth, A. K., & Bayne, T. (2022) · Nature Reviews Neuroscience · doi:10.1038/s41583-022-00587-4 · Cited by 880 · cited 1×
“Interestingly, some prominent theories of consciousness emphasize recurrent (i.e., top-down) signalling as critical for conscious perception ([123, 124], for a review see [125]).”
BackgroundSelf-citationPredictive Processing: A Canonical Cortical Computation
Keller, G. B., & Mrsic-Flogel, T. D. (2018) · Neuron · doi:10.1016/j.neuron.2018.10.003 · Cited by 836
Expectation in perceptual decision making: neural and computational mechanisms
Summerfield, C., & de Lange, F. P. (2014) · Nature Reviews Neuroscience · doi:10.1038/nrn3838 · Cited by 832
Dyna, an Integrated Architecture for Learning, Planning, and Reacting
Sutton, R. S. (1991) · ACM SIGART Bulletin · doi:10.1145/122344.122377 · Cited by 777 · cited 2×
“This idea of combining model-free and model-based planning methods (interpreted as iterative and amortized inference) in reinforcement learning also has a long history [59, 60].”
BackgroundMethodFast Backprojections from the Motion to the Primary Visual Area Necessary for Visual Awareness
Pascual-Leone, A., & Walsh, V. (2001) · Science · doi:10.1126/science.1057099 · Cited by 761 · cited 1×
“Interestingly, some prominent theories of consciousness emphasize recurrent (i.e., top-down) signalling as critical for conscious perception ([123, 124], for a review see [125]).”
BackgroundFriston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., ODoherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674
Friston, K. J., & Stephan, K. E. (2007) · Synthese · doi:10.1007/s11229-007-9237-y · Cited by 673
Deep reinforcement learning in a handful of trials using probabilistic dynamics models
Chua, K., Calandra, R., McAllister, R., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1805.12114 · Cited by 525
How prediction errors shape perception, attention, and motivation
den Ouden, H. E. M., Kok, P., & de Lange, F. P. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00548 · Cited by 512
Shape perception reduces activity in human primary visual cortex
Murray, S. O., Kersten, D., Olshausen, B. A., Schrater, P., & Woods, D. L. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.192579399 · Cited by 496
Kar, K., Kubilius, J., Schmidt, K., Issa, E. B., & DiCarlo, J. J. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0392-5 · Cited by 496
How neuroscience will change our view on consciousness
Lamme, V. A. F. (2010) · Cognitive Neuroscience · doi:10.1080/17588921003731586 · Cited by 487 · cited 1×
“…top-down signalling has long been recognised as playing several important functional roles [9]—for instance, in attentional modulation [10], in the goal-directed shaping of stimulus selection [11, 12], and in establishing recurrent loops that have been associated with conscious experience [13].”
BackgroundA review of predictive coding algorithms
Spratling, M. W. (2017) · Brain and Cognition · doi:10.1016/j.bandc.2015.11.003 · Cited by 437 · cited 1×
“predictive coding schemes which depart from this template [36], although in different ways to our proposed hybrid predictive coding model.”
MethodStimulus Predictability Reduces Responses in Primary Visual Cortex
Alink, A., Schwiedrzik, C. M., Kohler, A., Singer, W., & Muckli, L. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.3730-10.2010 · Cited by 435
Top-down predictions in the cognitive brain
Kveraga, K., Ghuman, A. S., & Bar, M. (2007) · Brain and Cognition · doi:10.1016/j.bandc.2007.06.007 · Cited by 423
Theories of Error Back-Propagation in the Brain
Whittington, J. C. R., & Bogacz, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.005 · Cited by 420
Evaluating the neurophysiological evidence for predictive processing as a model of perception
Walsh, K. S., McGovern, D. P., Clark, A., & O'Connell, R. G. (2020) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.14321 · Cited by 333
Repetition suppression and its contextual determinants in predictive coding
Auksztulewicz, R., & Friston, K. (2016) · Cortex · doi:10.1016/j.cortex.2015.11.024 · Cited by 307
A tutorial on the free-energy framework for modelling perception and learning
Bogacz, R. (2017) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.11.003 · Cited by 283 · cited 3×
“The resulting scheme can be interpreted as performing variational inference [29], an optimisation procedure that approximates Bayesian inference [38–40].”
BackgroundMethodWhittington, J. C. R., & Bogacz, R. (2017) · Neural Computation · doi:10.1162/neco_a_00949 · Cited by 278 · cited 2×
“Moreover, recent work has demonstrated that predictive coding provides a local approximation to backpropagation; the algorithm underwriting many of the recent successes in machine learning [84, 86, 130].”
BackgroundIs it an animal? Is it a human face? Fast processing in upright and inverted natural scenes
Rousselet, G. A., Macé, M. J.-M., & Fabre-Thorpe, M. (2003) · Journal of Vision · doi:10.1167/3.6.5 · Cited by 215
Predictive processing as a systematic basis for identifying the neural correlates of consciousness
Hohwy, J., & Seth, A. (2020) · Philosophy and the Mind Sciences · doi:10.33735/phimisci.2020.ii.64 · Cited by 170
Self-citationForms of prediction in the nervous system
Teufel, C., & Fletcher, P. C. (2020) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-0275-5 · Cited by 155
A predictive coding account of bistable perception - a model-based fMRI study
Weilnhammer, V., Stuke, H., Hesselmann, G., Sterzer, P., & Schmack, K. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005536 · Cited by 120
Predictive Coding Approximates Backprop Along Arbitrary Computation Graphs
Millidge, B., Tschantz, A., & Buckley, C. L. (2022) · Neural Computation · doi:10.1162/neco_a_01497 · Cited by 95
Self-citationA theory of learning to infer.
Dasgupta, I., Schulz, E., Tenenbaum, J. B., & Gershman, S. J. (2020) · Psychological Review · doi:10.1037/rev0000178 · Cited by 70
Predictive processing as an empirical theory for consciousness science
Seth, A. K., & Hohwy, J. (2021) · Cognitive Neuroscience · doi:10.1080/17588928.2020.1838467 · Cited by 47
Self-citationPredictive Coding: Towards a Future of Deep Learning beyond Backpropagation?
Millidge, B., Salvatori, T., Song, Y., Bogacz, R., & Lukasiewicz, T. (2022) · arXiv · doi:10.48550/arxiv.2202.09467 · Cited by 31
Self-citationFitting predictive coding to the neurophysiological data
Spratling, M. W. (2019) · Brain Research · doi:10.1016/j.brainres.2019.146313 · Cited by 23
A predictive-coding network that is both discriminative and generative
Sun, W., & Orchard, J. (2020) · Neural Computation · doi:10.1162/neco_a_01311 · Cited by 11
Research works printed but not linked: 91
- P Dayan, 1995, The Helmholtz machine, Neural computation, 7, 889, 10.1162/neco.1995.7.5.889 · DOI · Paper page
- K Friston, 2005, A theory of cortical responses, Philosophical transactions of the Royal Society B: Biological sciences, 360, 815, 10.1098/rstb.2005.1622 · DOI · Paper page
- R Weidner, 2009, Sources of top–down control in visual search, Journal of Cognitive Neuroscience, 21, 2100, 10.1162/jocn.2008.21173 · DOI · Paper page
- L Melloni, 2012, Interaction between bottom-up saliency and top-down control: how saliency maps are created in the human brain, Cerebral cortex, 22, 2943, 10.1093/cercor/bhr384 · DOI
- A Delorme, 2004, Interaction of top-down and bottom-up processing in the fast visual analysis of natural scenes, Cognitive Brain Research, 19, 103, 10.1016/j.cogbrainres.2003.11.010 · DOI · Paper page
- G Kreiman, 2020, Beyond the feedforward sweep: feedback computations in the visual cortex, Annals of the New York Academy of Sciences
- M Ahissar, 2004, The reverse hierarchy theory of visual perceptual learning, Trends in cognitive sciences, 8, 457, 10.1016/j.tics.2004.08.011 · DOI · Paper page
- R VanRullen, 2007, The power of the feed-forward sweep, Advances in Cognitive Psychology, 3, 167, 10.2478/v10053-008-0022-3 · DOI · Paper page
- PE Roland, 2010, Six principles of visual cortical dynamics, Frontiers in systems neuroscience, 4, 28, 10.3389/fnsys.2010.00028 · DOI
- K Rauss, 2013, What is bottom-up and what is top-down in predictive coding?, Frontiers in psychology, 4, 276, 10.3389/fpsyg.2013.00276 · DOI · Paper page
- N Gordon, 2017, Neural markers of predictive coding under perceptual uncertainty revealed with Hierarchical Frequency Tagging, Elife, 6, e22749, 10.7554/eLife.22749 · DOI · Paper page
- CL Buckley, 2017, The free energy principle for action and perception: A mathematical review, Journal of Mathematical Psychology, 81, 55, 10.1016/j.jmp.2017.09.004 · DOI · Paper page
- Millidge B, Seth AK, Buckley CL. Predictive coding: a theoretical and experimental review. arXiv preprint arXiv:210712979. 2021;.
- A Clark, 2015, Surfing uncertainty: Prediction, action, and the embodied mind
- Millidge B. Implementing predictive processing and active inference: Preliminary steps and results. PsyArXiv. 2019;. · DOI
- CW Fox, 2012, A tutorial on variational Bayesian inference, Artificial intelligence review, 38, 85, 10.1007/s10462-011-9236-8 · DOI · Paper page
- Millidge B, Tschantz A, Seth AK, Buckley C. Neural Kalman filtering. arXiv preprint arXiv:210210021. 2021;.
- Millidge B, Tschantz A, Seth AK, Buckley CL. Reinforcement Learning as Iterative and Amortised Inference. arXiv preprint arXiv:200610524. 2020;.
- Tschantz A, Millidge B, Seth AK, Buckley CL. Control as hybrid inference. arXiv preprint arXiv:200705838. 2020;.
- Marino J, Yue Y, Mandt S. Iterative amortized inference. In: International Conference on Machine Learning. PMLR; 2018. p. 3403–3412.
- van Bergen RS, Kriegeskorte N. Going in circles is the way forward: the role of recurrence in visual inference. arXiv preprint arXiv:200312128. 2020;. · DOI · Paper page
- C Keysers, 2001, The speed of sight, Journal of cognitive neuroscience, 13, 90, 10.1162/089892901564199 · DOI · Paper page
- T Carlson, 2013, Representational dynamics of object vision: the first 1000 ms, Journal of vision, 13, 1, 10.1167/13.10.1 · DOI · Paper page
- E Thunell, 2019, Memory for repeated images in rapid-serial-visual-presentation streams of thousands of images, Psychological science, 30, 989, 10.1177/0956797619842251 · DOI
- Kingma DP, Welling M. Auto-encoding variational bayes. arXiv preprint arXiv:13126114. 2013;.
- Doersch C. Tutorial on variational autoencoders. arXiv preprint arXiv:160605908. 2016;.
- Gershman S, Goodman N. Amortized inference in probabilistic reasoning. In: Proceedings of the annual meeting of the cognitive science society. vol. 36; 2014.
- Cremer C, Li X, Duvenaud D. Inference suboptimality in variational autoencoders. In: International Conference on Machine Learning. PMLR; 2018. p. 1078–1086.
- D Kahneman, 2011, Thinking, fast and slow
- Ghahramani Z, Beal MJ. Propagation algorithms for variational Bayesian learning. In: Advances in neural information processing systems; 2001. p. 507–513.
- Marino J, Piché A, Ialongo AD, Yue Y. Iterative amortized policy optimization. Advances in Neural Information Processing Systems. 2021;34.
- Millidge B, Tschantz A, Seth AK, Buckley CL. On the relationship between active inference and control as inference. In: International Workshop on Active Inference. Springer; 2020. p. 3–11. · DOI
- Schmidhuber J. Making the World Differentiable: On Using Self-Supervised Fully Recurrent Neural Networks for Dynamic Reinforcement Learning and Planning in Non-Stationary Environments. Technical Report: Technical University of Munich. 1990;. · DOI · Paper page
- Bengio Y, Scellier B, Bilaniuk O, Sacramento J, Senn W. Feedforward initialization for fast inference of deep generative networks is biologically plausible. arXiv preprint arXiv:160601651. 2016;.
- X Xie, 2003, Equivalence of backpropagation and contrastive Hebbian learning in a layered network, Neural computation, 15, 441, 10.1162/089976603762552988 · DOI · Paper page
- B Scellier, 2017, Equilibrium propagation: Bridging the gap between energy-based models and backpropagation, Frontiers in computational neuroscience, 11, 24, 10.3389/fncom.2017.00024 · DOI · Paper page
- RP Rao, 1999, Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects, Nature Neuroscience, 2, 79, 10.1038/4580 · DOI · Paper page
- D Xu, 2017, Symmetric predictive estimator for biologically plausible neural learning, IEEE Transactions on Neural Networks and Learning Systems, 29, 4140, 10.1109/TNNLS.2017.2756859 · DOI
- K Friston, 2012, The history of the future of the Bayesian brain, NeuroImage, 62, 1230, 10.1016/j.neuroimage.2011.10.004 · DOI · Paper page
- DC Knill, 1996, Perception as Bayesian inference, 10.1017/CBO9780511984037 · DOI · Paper page
- RT Cox, 1946, Probability, frequency and reasonable expectation, American journal of physics, 14, 1, 10.1119/1.1990764 · DOI · Paper page
- MJ Wainwright, 2008, Graphical models, exponential families, and variational inference, Foundations and Trends in Machine Learning, 1, 1, 10.1561/2200000001 · DOI · Paper page
- Odaibo S. Tutorial: Deriving the standard variational autoencoder (vae) loss function. arXiv preprint arXiv:190708956. 2019;.
- MD Hoffman, 2013, Stochastic variational inference, The Journal of Machine Learning Research, 14, 1303
- Ranganath R, Gerrish S, Blei D. Black box variational inference. In: Artificial intelligence and statistics. PMLR; 2014. p. 814–822.
- Millidge B, Tschantz A, Seth AK, Buckley CL. Relaxing the constraints on predictive coding models. arXiv preprint arXiv:201001047. 2020;.
- C Zhang, 2018, Advances in variational inference, IEEE transactions on pattern analysis and machine intelligence, 41, 2008, 10.1109/TPAMI.2018.2889774 · DOI · Paper page
- I Goodfellow, 2016, Deep learning
- Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980. 2014;.
- Kinghorn PF, Millidge B, Buckley CL. Preventing Deterioration of Classification Accuracy in Predictive Coding Networks. arXiv preprint arXiv:220807114. 2022;. · DOI
- K Doya, 2007, Bayesian brain: Probabilistic approaches to neural coding
- Wolpert DM, Ghahramani Z. Bayes rule in perception, action and cognition. The Oxford Companion to the Mind Oxford University Press (http://eprints.pascal-network.org/archive/00001354/). 2005;.
- MJ Ramstead, 2022, From generative models to generative passages: a computational approach to (neuro) phenomenology, Review of Philosophy and Psychology, 1
- K Suzuki, 2017, A deep-dream virtual reality platform for studying altered perceptual phenomenology, Scientific reports, 7, 1, 10.1038/s41598-017-16316-2 · DOI · Paper page
- MW Spratling, 2012, Unsupervised learning of generative and discriminative weights encoding elementary image components in a predictive coding model of cortical function, Neural computation, 24, 60, 10.1162/NECO_a_00222 · DOI · Paper page
- JS Johnson, 2005, The recognition of partially visible natural objects in the presence and absence of their occluders, Vision research, 45, 3262, 10.1016/j.visres.2005.06.007 · DOI
- D Wyatte, 2012, The limits of feedforward vision: Recurrent processing promotes robust object recognition when objects are degraded, Journal of Cognitive Neuroscience, 24, 2248, 10.1162/jocn_a_00282 · DOI · Paper page
- K Rajaei, 2019, Beyond core object recognition: Recurrent processes account for object recognition under occlusion, PLoS computational biology, 15, e1007001, 10.1371/journal.pcbi.1007001 · DOI · Paper page
- H Tang, 2018, Recurrent computations for visual pattern completion, Proceedings of the National Academy of Sciences, 115, 8835, 10.1073/pnas.1719397115 · DOI · Paper page
- S Grossberg, 2013, Adaptive Resonance Theory: How a brain learns to consciously attend, learn, and recognize a changing world, Neural networks, 37, 1, 10.1016/j.neunet.2012.09.017 · DOI · Paper page
- T Serre, 2007, A feedforward architecture accounts for rapid categorization, Proceedings of the national academy of sciences, 104, 6424, 10.1073/pnas.0700622104 · DOI · Paper page
- SL Brincat, 2006, Dynamic shape synthesis in posterior inferotemporal cortex, Neuron, 49, 17, 10.1016/j.neuron.2005.11.026 · DOI · Paper page
- DL Ringach, 1996, Spatial and temporal properties of illusory contours and amodal boundary completion, Vision research, 36, 3037, 10.1016/0042-6989(96)00062-4 · DOI · Paper page
- Y Sugase, 1999, Global and fine information coded by single neurons in the temporal visual cortex, Nature, 400, 869, 10.1038/23703 · DOI · Paper page
- Kim Y, Wiseman S, Miller A, Sontag D, Rush A. Semi-amortized variational autoencoders. In: International Conference on Machine Learning. PMLR; 2018. p. 2678–2687.
- Huang Y, Gornet J, Dai S, Yu Z, Nguyen T, Tsao D, et al. Neural Networks with Recurrent Generative Feedback. Advances in Neural Information Processing Systems. 2020;33.
- J Gordon, 2020, Combining deep generative and discriminative models for Bayesian semi-supervised learning, Pattern Recognition, 100, 107156, 10.1016/j.patcog.2019.107156 · DOI
- T Grootswagers, 2019, The representational dynamics of visual objects in rapid serial visual processing streams, NeuroImage, 188, 668, 10.1016/j.neuroimage.2018.12.046 · DOI
- SA Yoo, 2019, Feed-forward visual processing suffices for coarse localization but fine-grained localization in an attention-demanding context needs feedback processing, Plos one, 14, e0223166, 10.1371/journal.pone.0223166 · DOI
- Y Mohsenzadeh, 2018, Ultra-Rapid serial visual presentation reveals dynamics of feedforward and feedback processes in the ventral visual pathway, Elife, 7, e36329, 10.7554/eLife.36329 · DOI
- Furtak M, Mudrik L, Bola M. The forest, the trees, or both? Hierarchy and interactions between gist and object processing during perception of real-world scenes. PsyArXiv. 2021;. · DOI
- H Karimi-Rouzbahani, 2020, Perceptual difficulty modulates the direction of information flow in familiar face recognition, bioRxiv
- CJ Spoerer, 2020, Recurrent neural networks can explain flexible trading of speed and accuracy in biological vision, PLoS computational biology, 16, e1008215, 10.1371/journal.pcbi.1008215 · DOI · Paper page
- LA Riggs, 1953, The disappearance of steadily fixated visual test objects, JOSA, 43, 495, 10.1364/JOSA.43.000495 · DOI · Paper page
- Millidge B. Fixational Eye Movements: Data Augmentation for the Brain? PsyArXiv. 2019;. · DOI
- VA Lamme, 2000, The role of primary visual cortex (V1) in visual awareness, Vision research, 40, 1507, 10.1016/S0042-6989(99)00243-6 · DOI · Paper page
- AK Seth, 2021, Being you: A new science of consciousness
- J Hohwy, 2008, Predictive coding explains binocular rivalry: An epistemological review, Cognition, 108, 687, 10.1016/j.cognition.2008.05.010 · DOI · Paper page
- Lotter W, Kreiman G, Cox D. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv:160508104. 2016;.
- Song Y, Lukasiewicz T, Xu Z, Bogacz R. Can the Brain Do Backpropagation?—Exact Implementation of Backpropagation in Predictive Coding Networks. Advances in Neural Information Processing Systems. 2020;33.
- A Oliva, 2005, Neurobiology of attention, 251, 10.1016/B978-012375731-9/50045-8 · DOI · Paper page
- A Oliva, 2006, Building the gist of a scene: The role of global image features in recognition, Progress in brain research, 155, 23, 10.1016/S0079-6123(06)55002-2 · DOI · Paper page
- Kingma DP, Rezende DJ, Mohamed S, Welling M. Semi-supervised learning with deep generative models. arXiv preprint arXiv:14065298. 2014;.
- Rezende DJ, Mohamed S, Wierstra D. Stochastic backpropagation and approximate inference in deep generative models. In: International conference on machine learning. PMLR; 2014. p. 1278–1286.
- Kuleshov V, Ermon S. Deep hybrid models: Bridging discriminative and generative approaches. In: Proceedings of the Conference on Uncertainty in AI (UAI); 2017.
- Liu H, Abbeel P. Hybrid discriminative-generative training via contrastive learning. arXiv preprint arXiv:200709070. 2020;.
- V Garcia Satorras, 2019, Combining Generative and Discriminative Models for Hybrid Inference, Advances in Neural Information Processing Systems, 32, 13825
- Van de Ven GM, Tolias AS. Generative replay with feedback connections as a general strategy for continual learning. arXiv preprint arXiv:180910635. 2018;.
- GM van de Ven, 2020, Brain-inspired replay for continual learning with artificial neural networks, Nature communications, 11, 1, 10.1038/s41467-020-17866-2 · DOI · Paper page
- Schmidhuber J. Making the World Differentiable: On Using Self-Supervised Fully Recurrent Neural Networks for Dynamic Reinforcement Learning and Planning in Non-Stationary Environments; 1990. · DOI · Paper page
- K Friston, 2017, Active inference, curiosity and insight, Neural computation, 29, 2633, 10.1162/neco_a_00999 · DOI · Paper page
Cited by
Unpacking the complexities of consciousness: Theories and reflections
Mudrik, L., Boly, M., Dehaene, S., Fleming, S. M., Lamme, V., Seth, A., & Melloni, L. (2025) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2025.106053 · Cited by 81
Self-citationPredictive coding networks for temporal prediction
Millidge, B., Tang, M., Osanlouy, M., Harper, N. S., & Bogacz, R. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011183 · Cited by 44
“There are also multiple potential solutions to this problem – firstly, the cortex may implement both an iterative and an amortized feedforward pass solution simultaneously [87], and there is evidence for precisely this.”
BackgroundSelf-citationPredictive coding with spiking neurons and feedforward gist signaling
Lee, K., Dora, S., Mejias, J. F., Bohte, S. M., & Pennartz, C. M. A. (2024) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2024.1338280 · Cited by 14
“However, we note that SNN-PC is 523 spike-based and unsupervised, whereas the model in [82] is rate-based and uses a mix of 524 supervised and unsupervised learning.”
BackgroundMethodA survey on neuro-mimetic deep learning via predictive coding
Salvatori, T., Mali, A., Buckley, C. L., Lukasiewicz, T., Rao, R. P. N., Friston, K., & Ororbia, A. (2026) · Neural Networks · doi:10.1016/j.neunet.2025.108161 · Cited by 2
Self-citationClosed-form predictive coding via hierarchical Gaussian filters
Baskakovs, A., Estebe, S., Enevoldsen, K., Nielbo, K., Mathys, C., & Legrand, N. (2026) · arXiv · doi:10.48550/arxiv.2605.20293
“…et al., 2025, Innocenti et al., 2025], bidirectional and hybrid models unifying generative and discriminative pathways [Oliviers et al., 2025, Tscshantz et al., 2023], PC beyond the Gaussian assumption [Pinchetti et al., 2022], and associative memory architectures [Salvatori et al., 2021].”
Metadata
- Authors
- Tscshantz, A., Millidge, B., Seth, A. K., & Buckley, C. L.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2023-08-02
- Volume
- 19
- Issue
- 8
- Pages
- e1011280
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W4385481140
- Cited by
- 51
- References
- 54
- FWCI
- 7.12
- Type
- journal article
- Language
- en
- Update notices
- correction (2023-10-27)
- Open access
- gold
- Funders
- European Research Council (101012954)
- Canadian Institute for Advanced Research (Program in Brain, Mind, and Consciousness)
- Dr. Mortimer and Theresa Sackler Foundation
- School of Engineering and Informatics at the University of Sussex
- Biotechnology and Biological Sciences Research Council (BB/P022197/1, BB/s006338/1)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.