Bayesian Computation in Recurrent Neural Circuits
The DOI matches its Crossref record; 77 of the 78 references it cites were found in a registry or an index.
Abstract
A large number of human psychophysical results have been successfully explained in recent years using Bayesian models. However, the neural implementation of such models remains largely unclear. In this article, we show that a network architecture commonly used to model the cerebral cortex can implement Bayesian inference for an arbitrary hidden Markov model. We illustrate the approach using an orientation discrimination task and a visual motion detection task. In the case of orientation discrimination, we show that the model network can infer the posterior distribution over orientations and correctly estimate stimulus orientation in the presence of significant noise. In the case of motion detection, we show that the resulting model network exhibits direction selectivity and correctly computes the posterior probabilities over motion direction and position. When used to solve the well-known random dots motion discrimination task, the model generates responses that mimic the activities of evidence-accumulating neurons in cortical areas LIP and FEF. The framework we introduce posits a new interpretation of cortical activities in terms of log posterior probabilities of stimuli occurring in the natural world.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Message-passing inference in CSCG is computationally cheap and biologically plausible using simple integrate-and-fire neurons (23).”
From Space is a latent sequence: A theory of the hippocampus (2024) · cites this paper as [23]
In the citing paper
“Moreover, there are population coding mechanisms that could support explicit probabilistic computations (Zemel and Dayan, 1997; Sahani and Dayan, 2003; Rao, 2004; Ma et al., 2006; Eliasmith and Martens, 2011; Gershman and Beck, 2016).”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Rao, 2004
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444 · cited 1×
“We intend to explore the question of synaptic learning using biologically-plausible approximations to the expectation-maximization (EM) algorithm (Dempster, Laird, & Rubin, 1977).”
MethodHebb, D. O. (2005) · doi:10.4324/9781410612403 · Cited by 9,825
Distributed hierarchical processing in the primate cerebral cortex
Felleman, D. J., & Van Essen, D. C. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1 · Cited by 7,783 · cited 2×
“Such models would provide a better approximation to the multi-layered architecture of the cortex and the hierarchical connections that exist between cortical areas (Felleman & Van Essen, 1991).”
BackgroundRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 4×
“…question is how the suggested framework could be extended to allow neural implementation of hierarchical Bayesian inference in graphical models that are more general than hidden Markov models (cf. (Dayan et al., 1995; Dayan & Hinton, 1996; Hinton & Ghahramani, 1997; Rao & Ballard, 1997, 1999)).”
BackgroundMethodResultInfluentialSelf-citationAn introduction to hidden Markov models
Rabiner, L., & Juang, B. (1986) · IEEE ASSP Magazine · doi:10.1109/massp.1986.1165342 · Cited by 4,775 · cited 1×
“For standard hidden Markov models, the E-step in the EM algorithm is realized by the “forward-backward” algorithm (Rabiner & Juang, 1986).”
MethodTheoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Dayan, P., & Abbott, L. F. (2001) · Cited by 4,099
The Handbook of brain theory and neural networks
(1996) · Choice Reviews Online · doi:10.5860/choice.33-4484 · Cited by 3,870
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588
Parallel & distributed processing
Slusallek, P., Shirley, P., Mark, W., Stoll, G., & Wald, I. (2005) · ACM SIGGRAPH 2005 Courses on - SIGGRAPH '05 · doi:10.1145/1198555.1198750 · Cited by 2,568
Biophysics of Computation: Information Processing in Single Neurons
Koch, C. (1998) · Oxford University Press eBooks · doi:10.1093/oso/9780195104912.001.0001 · Cited by 2,283
Research works printed but not linked: 1
- 10.1037/0033-295X.106.2.261 · doi:10.1037/0033-295x.106.2.261 · Paper page
Cited by
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588
The free-energy principle: a rough guide to the brain?
Friston, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.04.005 · Cited by 1,831
Bayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
Shortlist B: A Bayesian model of continuous speech recognition.
Norris, D., & McQueen, J. M. (2008) · Psychological Review · doi:10.1037/0033-295x.115.2.357 · Cited by 686
Probabilistic brains: knowns and unknowns
Pouget, A., Beck, J. M., Ma, W. J., & Latham, P. E. (2013) · Nature Neuroscience · doi:10.1038/nn.3495 · Cited by 615
“Several groups have explored a variation of these ideas in which, rather than coding for the log probability that a feature is present, neurons code for the log probability that a feature takes on a particular value 38–42 .”
BackgroundDecision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
“One question under active debate is the way that uncertain information, such as belief states, might be represented in the activity of populations of neurons, and support the basic computations of Bayesian decision theory such as belief updating (equation 26) and the calculation of expected values (equation 22) (Zemel…”
BackgroundInfluentialBrain plasticity through the life span: Learning to learn and action video games
Bavelier, D., Green, C. S., Pouget, A., & Schrater, P. (2012) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-060909-152832 · Cited by 541
The Bayesian reader: Explaining word recognition as an optimal Bayesian decision process.
Norris, D. (2006) · Psychological Review · doi:10.1037/0033-295x.113.2.327 · Cited by 412
“Possible neural mechanisms to compute likelihood ratios are discussed in Gold and Shadlen (2001), while Rao (2004) shows how a recurrent network architecture can implement Bayesian inference for an arbitrary hidden Markov model.”
BackgroundElapsed Decision Time Affects the Weighting of Prior Probability in a Perceptual Decision Task
Hanks, T. D., Mazurek, M. E., Kiani, R., Hopp, E., & Shadlen, M. N. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.5613-10.2011 · Cited by 378
“Thus, it has been suggested that there is a probabilistic evaluation of sensory signals in the brain (Barlow, 1969; Carpenter and Williams, 1995; Zemel et al., 1998; Rao, 2004; Jazayeri and Movshon, 2006; Ma et al., 2006; Gold and Shadlen, 2007; Beck et al., 2008).”
BackgroundVisual adaptation: Neural, psychological and computational aspects
Clifford, C. W. G., Webster, M. A., Stanley, G. B., Stocker, A. A., Kohn, A., Sharpee, T. O., & Schwartz, O. (2007) · Vision Research · doi:10.1016/j.visres.2007.08.023 · Cited by 371
“Future work should investigate the implementation of such a computational framework in a biologically realizable architecture (e.g. Rao, 2004) and explore the behaviour of such networks under adaptation (e.g. Teich & Qian, 2003).”
BackgroundIntegrated information theory of consciousness: an updated account.
(2012) · PubMed · doi:10.4449/aib.v149i5.1388 · Cited by 370
Bayesian Spiking Neurons I: Inference
Deneve, S. (2008) · Neural Computation · doi:10.1162/neco.2008.20.1.91 · Cited by 313
“In contrast, previous models had proposed that firing rates explicitly represent probabilities (Mazurek et al., 2003; Rao, 2003; Zemel et al., 1998; Sahani & Dayan, 2003).”
BackgroundResultA computational framework for the study of confidence in humans and animals
Kepecs, A., & Mainen, Z. F. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0037 · Cited by 305
A tutorial introduction to Bayesian models of cognitive development
Perfors, A., Tenenbaum, J. B., Griffiths, T. L., & Xu, F. (2011) · Cognition · doi:10.1016/j.cognition.2010.11.015 · Cited by 304
“These issues are being overcome as the state of the art advances (see Rao, Olshausen, & Lewicki (2002) for some examples), but for the models most commonly used in cognitive science – perceptrons, multilayered recurrent networks, and Boltzmann machines – they remain a relevant concern.”
Neural Coding of Uncertainty and Probability
Ma, W. J., & Jazayeri, M. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071013-014017 · Cited by 290
“Explicit probability codes compose a third class of neural codes for uncertainty (Barlow & Levick 1969, Anderson 1994, Anastasio et al. 2000, Barber et al. 2003, Lee & Mumford 2003, Rao 2004, Deneve 2008).”
BackgroundNeural Variability and Sampling-Based Probabilistic Representations in the Visual Cortex
Orbán, G., Berkes, P., Fiser, J., & Lengyel, M. (2016) · Neuron · doi:10.1016/j.neuron.2016.09.038 · Cited by 255
“In this class of models, changes in response variability are directly linked to changes in average responses and thus do not serve as an independent information channel (Deneve, 2008; Ma et al., 2006; Rao, 2004; Zemel et al., 1998).”
BackgroundInfluentialInternal models in biological control
McNamee, D., & Wolpert, D. M. (2019) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-060117-105206 · Cited by 225
“These considerations have motivated the development of dynamical models of cortex with recurrent connectivity which approximate Bayesian inference [32, 34] though the characterization of this class of models from a computational point of view remains an ongoing challenge [39].”
BackgroundEmergence of complex cell properties by learning to generalize in natural scenes
Karklin, Y., & Lewicki, M. S. (2009) · Nature · doi:10.1038/nature07481 · Cited by 218
Noise as a Resource for Computation and Learning in Networks of Spiking Neurons
Maass, W. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2310593 · Cited by 198
“, [37], [78], and [79]) have shown that some elements of probabilistic inference that can be implemented through deterministic message passing (belief propagation) can, in principle, also be implemented by deterministic networks of spiking neurons (in fact, any deterministic computation, i.”
BackgroundWhat and where: A Bayesian inference theory of attention
Chikkerur, S., Serre, T., Tan, C., & Poggio, T. (2010) · Vision Research · doi:10.1016/j.visres.2010.05.013 · Cited by 197
“It was recently suggested that visual perception may be interpreted as a Bayesian inference process whereby top-down priors help disambiguate noisy bottom-up sensory input signals (Dayan, Hinton, & Neal, 1995; Dayan & Zemel, 1999; Dean, 2005; Epshtein, Lifshitz, & Ullman, 2008; Friston, 2003; George & Hawkins, 2005; H…”
InfluentialRao, R. P. N. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00146 · Cited by 191
“A neural model as sketched above for approximate Bayesian inference but using a linear recurrent network was first explored in (Rao, 2004).”
Self-citationSato, Y., Toyoizumi, T., & Aihara, K. (2007) · Neural Computation · doi:10.1162/neco.2007.19.12.3335 · Cited by 186
“Biologically plausible implementations of the method and their relations to the activity of neural circuits have also been investigated (Deneve, Latham, & Pouget, 2001; Rao, 2004).”
MethodBayesian processing of vestibular information
Laurens, J., & Droulez, J. (2007) · Biological Cybernetics · doi:10.1007/s00422-006-0133-1 · Cited by 177
“Other models have been proposed, based on a succesion of feedback loops (Lee and Mumford 2003; Rao 2004).”
BackgroundModelling trial-by-trial changes in the mismatch negativity
Lieder, F., Daunizeau, J., Garrido, M. I., Friston, K. J., & Stephan, K. E. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002911 · Cited by 168
“Previous work along these lines has assumed that the support of probability distributions is partitioned into small bins and that each bin’s probability mass is represented by the firing rate of dedicated neurons [77,78], or that probability densities are approximated by a linear combination of basis functions [79].”
BackgroundOrganizing probabilistic models of perception
Ma, W. J. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.08.010 · Cited by 171
Bayes in the Brain—On Bayesian Modelling in Neuroscience
Colombo, M., & Seriès, P. (2012) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axr043 · Cited by 164
Bayesian inference and attentional modulation in the visual cortex
Rao, R. P. N. (2005) · NeuroReport · doi:10.1097/01.wnr.0000183900.92901.fc · Cited by 161
“[12–18]). We extend the model in [ 19 ] to implement the belief propagation equations above in a neural circuit (Fig. 1b).”
MethodInfluentialSelf-citationNeural processing as causal inference
Lochmann, T., & Deneve, S. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.05.018 · Cited by 158
“Recent approaches used HMMs to understand how biochemical mechanisms of phototransduction [11], single synapses [12 ,13 ], single neurons [8 ,14] or recurrent neural networks [15–17,18 ,19] could optimally”
Perceptual decision making: drift-diffusion model is equivalent to a Bayesian model
Bitzer, S., Park, H., Blankenburg, F., & Kiebel, S. J. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00102 · Cited by 157
“One approach of addressing this question has been termed perceptual decision making where the typical experiment uses a two-alternative forced choice task to judge differences in perceptual features (e.g., Newsome et al., 1989; Heekeren et al., 2004; Summerfield et al., 2006).”
BackgroundMethodInfluentialArtificial Cognitive Systems: From VLSI Networks of Spiking Neurons to Neuromorphic Cognition
Indiveri, G., Chicca, E., & Douglas, R. J. (2009) · Cognitive Computation · doi:10.1007/s12559-008-9003-6 · Cited by 145
“Several recent studies have considered how single neurons or their networks could implement belief propagation [52, 55], or of how they could perform probabilistic computations in”
BackgroundDenève, S., Duhamel, J.-R., & Pouget, A. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3985-06.2007 · Cited by 137
“Several coding schemes have been explored previously (Zemel et al., 1998; Weiss and Fleet, 2002; Bar-ber et al., 2003; Mazurek et al., 2003; Sahani and Dayan, 2003; Ma et al., 2006), and several groups are exploring how to use these schemes for optimal filtering (Deneve, 2004; Rao, 2004).”
BackgroundDynamics of attentional selection under conflict: Toward a rational Bayesian account.
Yu, A. J., Dayan, P., & Cohen, J. D. (2009) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/a0013553 · Cited by 144
“…may encode probabilistic information about the sensory world (Anderson , 1995; Deneve, 2005; Gold & Shadlen, 2002; Ma, Beck, Latham, & Pouget, 2006; Rao, 2004; Sahani & Dayan, 2003; Weiss & Fleet, 2002; Yu, 2007; Zemel, Dayan, & Pouget, 1998), given the noisy, stochastic nature of sensory…”
BackgroundToward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119
STDP Installs in Winner-Take-All Circuits an Online Approximation to Hidden Markov Model Learning
Kappel, D., Nessler, B., & Maass, W. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003511 · Cited by 119
“These studies targeted the implementation of Bayesian filtering [56,57], capturing the forward message of the belief propagation algorithm in a rate-based neural code, or using a two-state HMM to capture the dynamics of single neurons [58,59].”
MethodGeorge, D., Rikhye, R. V., Gothoskar, N., Guntupalli, J. S., Dedieu, A., & Lázaro-Gredilla, M. (2021) · Nature Communications · doi:10.1038/s41467-021-22559-5 · Cited by 109
Pecevski, D., Buesing, L., & Maass, W. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002294 · Cited by 109
Spiking networks for Bayesian inference and choice
Ma, W. J., Beck, J. M., & Pouget, A. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.07.004 · Cited by 113
“) More complex neural network schemes have also been designed for (hierarchical) Bayesian inference [27,29,42,52,53], but these were not meant to describe actual neurons, or there is little experimental data to test their validity.”
BackgroundInfluentialSpike-Based Population Coding and Working Memory
Boerlin, M., & Denève, S. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001080 · Cited by 102
Evidence for surprise minimization over value maximization in choice behavior
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., Kronbichler, M., & Friston, K. (2015) · Scientific Reports · doi:10.1038/srep16575 · Cited by 96
Going in circles is the way forward: the role of recurrence in visual inference
van Bergen, R. S., & Kriegeskorte, N. (2020) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2020.11.009 · Cited by 93
Shifting the spotlight of attention: evidence for discrete computations in cognition
Buschman, T. J., & Miller, E. K. (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00194 · Cited by 87
A Bayesian Perspective on Sensory and Cognitive Integration in Pain Perception and Placebo Analgesia
Anchisi, D., & Zanon, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0117270 · Cited by 84
Optimal decision making in heterogeneous and biased environments
Moran, R. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-014-0669-3 · Cited by 100
Image interpretation by a single bottom-up top-down cycle
Epshtein, B., Lifshitz, I., & Ullman, S. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0800968105 · Cited by 86
State-Dependent Computation Using Coupled Recurrent Networks
Rutishauser, U., & Douglas, R. J. (2008) · Neural Computation · doi:10.1162/neco.2008.03-08-734 · Cited by 86
“In theory, these recurrent circuits exhibit a variety of interesting computations (Abbott, 1994; Douglas, Koch, Mahowald, Martin, & Suarez, 1995; Ben-Yishai, Bar-Or, & Sompolinsky, 1995; Hahnloser, Sarpeshkar, Mahowald, Douglas, & Seung, 2000; Zhaoping, 2001; Pouget, Zhang, Deneve, & Latham, 1998; Machens, Romo, & Bro…”
BackgroundGlaze, C. M., Filipowicz, A. L. S., Kable, J. W., Balasubramanian, V., & Gold, J. I. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0297-4 · Cited by 92
“(1 ) Pr( )) (19) n z n n n z z n n z n n n z z n n”
Inference, Attention, and Decision in a Bayesian Neural Architecture
Yu, A. J., & Dayan, P. (2004) · MPG.PuRe (Max Planck Society) · Cited by 76
Perceptual Inference Predicts Contextual Modulations of Sensory Responses
Lochmann, T., Ernst, U. A., & Denève, S. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.0817-11.2012 · Cited by 83
“, 1998) or direct representations of probabilities in firing rates (Rao, 2004).”
Fading memory and time series prediction in recurrent networks with different forms of plasticity
Lazăr, A., Pipa, G., & Triesch, J. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.020 · Cited by 94
“plement specific computational functions such as maximum likelihood estimation or Bayesian inference (e.g., Deneve, Latham, & Pouget, 1999; Rao, 2004 )?”
Computational Models of Spoken Word Recognition
Magnuson, J. S., Mirman, D., & Harris, H. D. (2018) · Cambridge University Press eBooks · doi:10.1017/cbo9781139029377.006 · Cited by 80
Bayesian models of child development
Gopnik, A., & Bonawitz, E. B. (2014) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1330 · Cited by 88
Hierarchical Bayesian Inference in Networks of Spiking Neurons
Rao, R. P. N. (2004) · neural information processing systems · Cited by 81
Self-citationInference in the Brain: Statistics Flowing in Redundant Population Codes
Pitkow, X., & Angelaki, D. E. (2017) · Neuron · doi:10.1016/j.neuron.2017.05.028 · Cited by 73
Computing with Continuous Attractors: Stability and Online Aspects
Wu, S., & Амари, Ш.-И. (2005) · Neural Computation · doi:10.1162/0899766054615626 · Cited by 69
“It is widely believed that Bayesian inference should be used in neural systems to cope with typical inputs in the natural environment that are both noisy and highly structured (see, e.g., Mumford, 1992; Dayan, Hinton, Neal, & Zemel, 1995; Zemel, Dayan, & Pouget, 1998; Zhang, Ginzburg, McNaughton, & Sejnowski, 1998; Ba…”
BackgroundTemporal dynamics of decision-making during motion perception in the visual cortex
Grossberg, S., & Pilly, P. K. (2008) · Vision Research · doi:10.1016/j.visres.2008.02.019 · Cited by 68
“Several authors (Gold & Shadlen, 2001, 2007; Jazayeri & Movshon, 2006; Ma, Beck, Latham, & Pouget, 2006; Rao, 2004) have suggested that these data illustrate Bayesian inference in the brain.”
Bayesian Cognitive Science, Unification, and Explanation
Colombo, M., & Hartmann, S. (2017) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axv036 · Cited by 63
“Josh Tenenbaum and colleagues, for example, emphasize that Bayesian decision theory provides us with a ‘unifying mathematical language for framing cognition as the solution to inductive problems’ (Tenenbaum et al. [2011], p.”
BackgroundHartmann, C., Lazăr, A., Nessler, B., & Triesch, J. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004640 · Cited by 70
Bayesian modeling of dynamic motion integration
Montagnini, A., Mamassian, P., Perrinet, L., Castet, É., & Masson, G. S. (2007) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2007.10.013 · Cited by 62
“An interesting model of a cortical recurrent neural network implementing dynamic Bayesian inference has been proposed by Rao (2004) .”
Legenstein, R., & Maass, W. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003859 · Cited by 58
“Several non-spiking models for Bayesian filtering have been proposed previously [14,49,50].”
MethodA Neural Computation for Visual Acuity in the Presence of Eye Movements
Pitkow, X., Sompolinsky, H., & Meister, M. (2007) · PLoS Biology · doi:10.1371/journal.pbio.0050331 · Cited by 58
“One study of particular relevance also describes a neural network for approximately Bayesian decoding of arbitrary hidden Markov processes [90].”
BackgroundPredation and the Origin of Neurones
Monk, T., & Paulin, M. G. (2014) · Brain, Behavior and Evolution · doi:10.1159/000368177 · Cited by 56
Belief Propagation in Networks of Spiking Neurons
Steimer, A., Maass, W., & Douglas, R. (2009) · Neural Computation · doi:10.1162/neco.2009.08-08-837 · Cited by 54
“, 2006), or depend on rough approximations (Rao, 2004).”
BackgroundResultInfluentialAmbiguity and invariance: two fundamental challenges for visual processing
Rust, N. C., & Stocker, A. A. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.04.013 · Cited by 62
“[19,20]) formed; how prior probabilities can be represented [21]; how likelihood functions can be multiplied (i.”
BackgroundModeling feature-based attention as an active top-down inference process
Hamker, F. H. (2006) · Biosystems · doi:10.1016/j.biosystems.2006.03.010 · Cited by 59
“…demonstrated for several times (Grossberg, 1980; Mumford, 1992; Ullman, 1995; Tononi et al., 1992; Tsotsos et al., 1995; Rao and Ballard, 1999; Rao, 1999; Hamker, 1999; Engel et al., 2001; Hamker and Worcester, 2002; Corchs and Deco, 2002; Hochstein and Ahissar, 2002; Rao, 2004; Hamker, 2004b).”
BackgroundTestolin, A., & Zorzi, M. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00073 · Cited by 59
Attention in a Bayesian Framework
Whiteley, L., & Sahani, M. (2012) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2012.00100 · Cited by 52
A Neural Implementation of the Kalman Filter
Wilson, R. C., & Finkel, L. H. (2009) · neural information processing systems · Cited by 61
Bayesian Inference with Spiking Neurons
Denève, S. (2015) · Springer eBooks · doi:10.1007/978-1-4614-6675-8_568 · Cited by 50
Cortical Circuitry Implementing Graphical Models
Litvak, S., & Ullman, S. (2009) · Neural Computation · doi:10.1162/neco.2009.05-08-783 · Cited by 52
“This approach is different from that of several past models (Beck & Pouget, 2007 ; Deneve, 2008a ; Ma et al, 2006 ; Ott & Stoop, 2006 ; Rao, 2004a , 2005b ), which suggested mapping a single neuronal unit to one element (a variable or a state) of the graphical model.”
MethodResultInfluentialExact Inferences in a Neural Implementation of a Hidden Markov Model
Beck, J. M., & Pouget, A. (2007) · Neural Computation · doi:10.1162/neco.2007.19.5.1344 · Cited by 51
“…firing rate u i ( t ) of neuron i is related to the probability that the stimulus takes on some value θ i through some monotonically increasing function, where f ( x ) is typically equal to the identity, a convolution (Anderson, 1994), or a log transform (Rao, 2004; Yu & Dayan, 2005; Den`eve, 2005).”
BackgroundMethodInfluentialObject Motion Computation for the Initiation of Smooth Pursuit Eye Movements in Humans
Wallace, J. M., Stone, L. S., & Masson, G. S. (2005) · Journal of Neurophysiology · doi:10.1152/jn.01042.2004 · Cited by 47
An Integrated Information Theory of Consciousness
Tononi, G. (2009) · Elsevier eBooks · doi:10.1002/9781119132363.ch17 · Cited by 48
Toward the Next Generation of Retinal Neuroprosthesis: Visual Computation with Spikes
Yu, Z., Liu, J. K., Jia, S., Zhang, Y., Zheng, Y., Tian, Y., & Huang, T. (2020) · Engineering · doi:10.1016/j.eng.2020.02.004 · Cited by 50
Post-injury pain and behaviour: a control theory perspective
Seymour, B., Crook, R. J., & Chen, Z. S. (2023) · Nature reviews. Neuroscience · doi:10.1038/s41583-023-00699-5 · Cited by 45
The vestibular component in out-of-body experiences: a computational approach
Schwabe, L., & Blanke, O. (2008) · Frontiers in Human Neuroscience · doi:10.3389/neuro.09.017.2008 · Cited by 45
“…1999) as it is supported by experimental (Ernst and Banks, 2002; Körding and Wolpert, 2004; MacNeilage et al., 2007; van Beers et al., 1996) and theoretical studies relating it to neuronal activations (Denève et al., 2007; Huys et al., 2007; Ma et al., 2006; Rao, 2004; Sahani and Dayan, 2003).”
BackgroundInfluentialProbabilistic population codes and the exponential family of distributions
Beck, J. M., Ma, W. J., Latham, P. E., & Pouget, A. (2007) · Progress in brain research · doi:10.1016/s0079-6123(06)65032-2 · Cited by 40
“Moreover, simple addition was shown to be optimal regardless of the variability of each cue, i.e., unlike other proposed cue combination schemes (Rao, 2004; Navalpakkam and Itti, 2005), this scheme does not require that the weights of the linear combination be adjusted on a trial-by-trial basis.”
BackgroundResultInfluentialIntegrated Bayesian models of learning and decision making for saccadic eye movements
Brodersen, K. H., Penny, W. D., Harrison, L. M., Daunizeau, J., Ruff, C. C., Duzel, E., Friston, K. J., & Stephan, K. E. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.08.007 · Cited by 40
“Furthermore, the way the brain might implement Bayesian inference can be expressed in alternative mathematical ways (Jazayeri & Movshon, 2006; Ma, Beck, Latham, & Pouget, 2006; Rao, 2004).”
Bayesian Filtering in Spiking Neural Networks: Noise, Adaptation, and Multisensory Integration
Bobrowski, O., Meir, R., & Eldar, Y. C. (2008) · Neural Computation · doi:10.1162/neco.2008.01-08-692 · Cited by 45
“Rao’s networks ( Rao, 2004 ) use linear dynamics and discrete time to approximately compute the log-posterior distributions from noisy continuous inputs (rather than actual spike trains).”
BackgroundInfluentialVariational Learning for Recurrent Spiking Networks
Rezende, D. J., Wierstra, D., & Wulfram Gerstner, B. (2011) · neural information processing systems · Cited by 40
Avram, M., Hennig‐Fast, K., Bao, Y., Pöppel, E., Reiser, M. F., Blautzik, J., Giordano, J., & Gutyrchik, E. (2014) · BMC Neuroscience · doi:10.1186/1471-2202-15-39 · Cited by 54
“As well, given that (a) most decisions, inclusive of potentially moral judgments involve some degree of Bayesian processing [32,33]; (b) such processing involves recollection of circumstance, effect and potential consequences in orientation to self, others and situations [2,5,34], and (c) learning and memory have been…”
BackgroundHierarchical Bayesian Inference and Learning in Spiking Neural Networks
Guo, S., Yu, Z., Deng, F., Hu, X., & Chen, F. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2768554 · Cited by 45
“In summary, Rao and Ballard’s work is based on the theory of predictive coding.”
InfluentialReward Optimization in the Primate Brain: A Probabilistic Model of Decision Making under Uncertainty
Huang, Y., & Rao, R. P. N. (2013) · PLoS ONE · doi:10.1371/journal.pone.0053344 · Cited by 44
Self-citationTowards a Neural Implementation of Causal Inference in Cue Combination
Ma, W. J., & Rahmati, M. (2013) · Multisensory Research · doi:10.1163/22134808-00002407 · Cited by 40
“…to neural activity (Anastasio et al., 2000; Anderson, 1994; Barlow, 1969; Berkes et al., 2011; Deneve, 2008; Fiser et al., 2010; Hoyer and Hyvarinen, 2003; Jazayeri and Movshon, 2006; Ma et al., 2006; Pouget et al., 2003; Rao, 2004; Shi et al., 2010; Vilares and Kording, 2011; Zemel et al., 1998).”
BackgroundInfluentialHuman inference reflects a normative balance of complexity and accuracy
Tavoni, G., Doi, T., Pizzica, C., Balasubramanian, V., & Gold, J. I. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01357-z · Cited by 35
“In this setting, the Bayesian ideal observer estimates the full distribution of the source in terms of two quantities: (1) the conditional probability p(rt∣x1:t) of the run-length rt, which is the number of time steps elapsed at time t since the last inferred change-point in the source; and (2) the probability p(μt∣rt…”
BackgroundMethodInfluentialPark, I. M., & Pillow, J. W. (2026) · PLoS Computational Biology · doi:10.1101/178418 · Cited by 32
“A substantial literature has sought to alter or expand the original efficient coding hypothesis [5, 8–18], and a large number of papers have considered optimal codes in the context of Bayesian inference [19–26].”
Error-based analysis of optimal tuning functions explains phenomena observed in sensory neurons
Yaeli, S., & Meir, R. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00130 · Cited by 37
Emergent Inference of Hidden Markov Models in Spiking Neural Networks Through Winner-Take-All
Yu, Z., Guo, S., Deng, F., Yan, Q., Huang, K., Liu, J. K., & Chen, F. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2871144 · Cited by 37
“Rao [3] built the relationship between the dynamic equation of recurrent neural networks and the inference equation of HMMs, and suggested that the dynamic process of a recurrent neural network is a process of probabilistic inference.”
BackgroundExploiting noise as a resource for computation and learning in spiking neural networks
Ma, G., Yan, R., & Tang, H. (2023) · Patterns · doi:10.1016/j.patter.2023.100831 · Cited by 59
“Some studies 31–34 have presented smallnetworksofnoisyspiking neuronsthatcan performprobabi-listic inference.”
BackgroundThe Visual System's Internal Model of the World
Lee, T. S. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2434601 · Cited by 31
“Neural implementation of exact belief propagation usually requires more elaborate dendritic computations [73], [107].”
BackgroundThe development of Bayesian integration in sensorimotor estimation
Chambers, C., Sokhey, T., Gaebler-Spira, D. J., & Kording, K. P. (2018) · Journal of Vision · doi:10.1167/18.12.8 · Cited by 33
“However, Bayesian behavior simply represents optimal behavior under uncertainty and there are ways of generating optimal behavior that do not explicitly implement Bayesian computation (Rao, 2004).”
BackgroundProbabilistic Computation in Spiking Populations
Zemel, R. S., Huys, Q. J. M., Natarajan, R., & Dayan, P. (2004) · UCL Discovery (University College London) · Cited by 31
Complex Probabilistic Inference
Gershman, S. J., & Beck, J. M. (2017) · Wiley eBooks · doi:10.1002/9781119159193.ch33 · Cited by 35
“For example, Rao [64] proposed a neural implementation of the sum-product message passing algorithm implemented in the log probability domain.”
BackgroundMethodBayesian Integration of Information in Hippocampal Place Cells
Madl, T., Franklin, S., Chen, K., Montaldi, D., & Trappl, R. (2014) · PLoS ONE · doi:10.1371/journal.pone.0089762 · Cited by 32
Space is a latent sequence: A theory of the hippocampus
Raju, R. V., Guntupalli, J. S., Zhou, G., Wendelken, C., Lázaro-Gredilla, M., & George, D. (2024) · Science Advances · doi:10.1126/sciadv.adm8470 · Cited by 25
Changes in expectation consequent on experience, modeled by a simple, forgetful neural circuit
Anderson, A. J., & Carpenter, R. H. S. (2006) · Journal of Vision · doi:10.1167/6.8.5 · Cited by 30
“…aspects of decision making (Carpenter & Williams, 1995; Reddi et al., 2003) and can be implemented using plausible neural circuits (Gold & Shadlen, 2001; Rao, 2004), it may be that the brain simply calculates the most likely estimate of probability given the target’s previous pattern of appearance.”
BackgroundNeurons as Monte Carlo Samplers: Bayesian Inference and Learning in Spiking Networks
Huang, Y., & Rao, R. P. N. (2014) · neural information processing systems · Cited by 25
Self-citationWhat can neurons do for their brain? Communicate selectivity with bursts
Balduzzi, D., & Tononi, G. (2013) · Theory in Biosciences · doi:10.1007/s12064-012-0165-0 · Cited by 26
“Recent approaches have focused on Bayesian models [14–17] where, typically, the number of spikes outputted by a neuron or the likelihood of a neuron outputting spikes corresponds to the probability of some event.”
BackgroundInformation transmission with spiking Bayesian neurons
Lochmann, T., & Denève, S. (2008) · New Journal of Physics · doi:10.1088/1367-2630/10/5/055019 · Cited by 26
“Closely related models were presented in [15, 16].”
Neural Models of Bayesian Belief Propagation
Rao, R. P. N. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/1535.003.0017 · Cited by 27
“Figure 11.5 Network Output for a Moving Stimulus (from [ 30 ]).”
BackgroundMethodInfluentialSelf-citationHow Prior Probability Influences Decision Making: A Unifying Probabilistic Model
Huang, Y., Hanks, T. D., Shadlen, M. N., Friesen, A. L., & Rao, R. P. N. (2012) · neural information processing systems · Cited by 35
Self-citation
Metadata
- Authors
- Rao, R. P. N.
- Affiliations
- University of Washington (US)
- Corresponding author
- Rao, Rajesh P. N.
- ORCID
- Rao, Rajesh P. N.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2004-01-01
- Volume
- 16
- Issue
- 1
- Pages
- 1-38
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2133861842
- Cited by
- 252
- References
- 78
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 5 · 2023: 8 · 2022: 5 · 2021: 8 · 2020: 8 · 2019: 5 · 2018: 12 · 2017: 8 · 2016: 10 · 2015: 12 · 2014: 20 · 2013: 10 · 2012: 16
- FWCI
- 9.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- Bayesian inference, recurrent neural circuits, hidden Markov models, orientation discrimination, visual motion detection, random dot motion, direction selectivity, posterior probability estimation, evidence accumulation, lateral intraparietal area, frontal eye field
- MeSH terms
- Action Potentials, Animals, Bayes Theorem, Cerebral Cortex, Discrimination Learning, Humans, Markov Chains, Motion Perception, Nerve Net, Neural Pathways, Synaptic Transmission, Neurons, Orientation, Pattern Recognition, Visual, Photic Stimulation, Reaction Time, Visual Fields, Artifacts, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.