Bayesian Computation in Recurrent Neural Circuits
The DOI matches its Crossref record; 77 of the 78 references it cites were found in a registry or an index.
Abstract
A large number of human psychophysical results have been successfully explained in recent years using Bayesian models. However, the neural implementation of such models remains largely unclear. In this article, we show that a network architecture commonly used to model the cerebral cortex can implement Bayesian inference for an arbitrary hidden Markov model. We illustrate the approach using an orientation discrimination task and a visual motion detection task. In the case of orientation discrimination, we show that the model network can infer the posterior distribution over orientations and correctly estimate stimulus orientation in the presence of significant noise. In the case of motion detection, we show that the resulting model network exhibits direction selectivity and correctly computes the posterior probabilities over motion direction and position. When used to solve the well-known random dots motion discrimination task, the model generates responses that mimic the activities of evidence-accumulating neurons in cortical areas LIP and FEF. The framework we introduce posits a new interpretation of cortical activities in terms of log posterior probabilities of stimuli occurring in the natural world.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Message-passing inference in CSCG is computationally cheap and biologically plausible using simple integrate-and-fire neurons (23).”
From Space is a latent sequence: A theory of the hippocampus (2024) · cites this paper as [23]
In the citing paper
“Moreover, there are population coding mechanisms that could support explicit probabilistic computations (Zemel and Dayan, 1997; Sahani and Dayan, 2003; Rao, 2004; Ma et al., 2006; Eliasmith and Martens, 2011; Gershman and Beck, 2016).”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Rao, 2004
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444 · cited 1×
“We intend to explore the question of synaptic learning using biologically-plausible approximations to the expectation-maximization (EM) algorithm (Dempster, Laird, & Rubin, 1977).”
MethodHebb, D. O. (2005) · doi:10.4324/9781410612403 · Cited by 9,825
Distributed hierarchical processing in the primate cerebral cortex
Felleman, D. J., & Van Essen, D. C. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1 · Cited by 7,783 · cited 2×
“Such models would provide a better approximation to the multi-layered architecture of the cortex and the hierarchical connections that exist between cortical areas (Felleman & Van Essen, 1991).”
BackgroundRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 4×
“…question is how the suggested framework could be extended to allow neural implementation of hierarchical Bayesian inference in graphical models that are more general than hidden Markov models (cf. (Dayan et al., 1995; Dayan & Hinton, 1996; Hinton & Ghahramani, 1997; Rao & Ballard, 1997, 1999)).”
BackgroundMethodResultInfluentialSelf-citationAn introduction to hidden Markov models
Rabiner, L., & Juang, B. (1986) · IEEE ASSP Magazine · doi:10.1109/massp.1986.1165342 · Cited by 4,775 · cited 1×
“For standard hidden Markov models, the E-step in the EM algorithm is realized by the “forward-backward” algorithm (Rabiner & Juang, 1986).”
MethodTheoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Dayan, P., & Abbott, L. F. (2001) · Cited by 4,099
The Handbook of brain theory and neural networks
(1996) · Choice Reviews Online · doi:10.5860/choice.33-4484 · Cited by 3,870
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588
Parallel & distributed processing
Slusallek, P., Shirley, P., Mark, W., Stoll, G., & Wald, I. (2005) · ACM SIGGRAPH 2005 Courses on - SIGGRAPH '05 · doi:10.1145/1198555.1198750 · Cited by 2,568
Biophysics of Computation: Information Processing in Single Neurons
Koch, C. (1998) · Oxford University Press eBooks · doi:10.1093/oso/9780195104912.001.0001 · Cited by 2,283
Research works printed but not linked: 1
- 10.1037/0033-295X.106.2.261 · doi:10.1037/0033-295x.106.2.261 · Paper page
Cited by
Items 101–200 of 251
Neural network learning of optimal Kalman prediction and control
Linsker, R. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.05.002 · Cited by 26
“The blending of ‘bottom-up’ sensory input and ‘top-down’ model-driven expectations has been discussed in the context of Bayesian inference and generative models, and various neural network (NN) algorithms are motivated by, approximate, or perform a portion of, the Bayesian inference process (e.g., George & Hawkins, 20…”
MethodB, T. J. S. A., Poizner, H., LYNCH, G. S., Gepshtein, S., & Greenspan, R. J. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2314297 · Cited by 28
“This suggests either that nature is performing Bayesian inference on joint probability distributions, which requires immense memory resources, or some approximate scheme that approaches optimal performance [22], [23].”
BackgroundRecognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks
Bitzer, S., & Kiebel, S. J. (2012) · Biological Cybernetics · doi:10.1007/s00422-012-0490-x · Cited by 25
“In computational neuroscience, models of recurrently connected networks of neurons, which optimally estimate dynamically changing states from noisy observations, have recently been proposed (Rao 2004; Denève et al. 2007; Natarajan et al. 2008; Wilson and Finkel 2009; Boerlin and Denève 2011).”
BackgroundInfluentialEliasmith, C. (2007) · Elsevier eBooks · doi:10.1016/b978-044451540-7/50028-1 · Cited by 28
“…and computational neuroscientists have been taking seriously the idea that neural systems must perform some kind of statistical inference in order to operate effectively in such an environment (Eliasmith and Anderson 2003; Kording and Wolpert 2004; Rao 2004; Deneve and Pouget in press).”
BackgroundXiong, A., & Proctor, R. W. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01270 · Cited by 35
“Rao (2004) illustrated how the Bayesian model could be implemented with neural networks by feedforward and recurrent connections, showing that for both perception and decision-making tasks the resulting network exhibits direction selectivity and computes posterior error corrections.”
BackgroundLearning Cognitive Map Representations for Navigation by Sensory–Motor Integration
Zhao, D., Zhang, Z., Lu, H., Cheng, S., Si, B., & Feng, X. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2977999 · Cited by 25
Optimal Change-Detection and Spiking Neurons
Yu, A. J. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0198 · Cited by 22
Inferring figure-ground using a recurrent integrate-and-fire neural circuit
Baek, K., & Sajda, P. (2005) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2005.847388 · Cited by 22
“However, in this previous model, the network nodes do not represent individual neurons and thus, it is difficult to relate their responses and implied representations to those of real cortical neurons.”
BackgroundHippocampus as unitary coherent particle filter
Fox, C., & Prescott, T. J. (2010) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596681 · Cited by 25
“Neural HMM mappings may be subsequently be constructed [14] so that the activation of the CA3 cells can represent the location posteriors.”
BackgroundTutorial Review of Bio-Inspired Approaches to Robotic Manipulation for Space Debris Salvage
Ellery, A. A. (2020) · Biomimetics · doi:10.3390/biomimetics5020019 · Cited by 24
Bayesian Inference and Online Learning in Poisson Neuronal Networks
Huang, Y., & Rao, R. P. N. (2016) · Neural Computation · doi:10.1162/neco_a_00851 · Cited by 23
Self-citationInterlimb Generalization of Learned Bayesian Visuomotor Prior Occurs in Extrinsic Coordinates
Hewitson, C. L., Sowman, P. F., & Kaplan, D. M. (2018) · eNeuro · doi:10.1523/eneuro.0183-18.2018 · Cited by 23
“Mounting neural, behavioral, and computational evidence suggests that the brain encodes probability distributions and performs probabilistic or Bayesian inference (Rao et al., 2002; Knill and Pouget, 2004; Rao, 2004; Ma et al., 2006; Doya, 2007; Pouget et al., 2013).”
BackgroundSequential Bayesian Decoding with a Population of Neurons
Wu, S., Chen, D., Niranjan, M., & Амари, Ш.-И. (2003) · Neural Computation · doi:10.1162/089976603765202631 · Cited by 24
Why aren’t all deep superior colliculus neurons multisensory? A Bayes’ ratio analysis
Colonius, H., & Diederich, A. (2004) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.4.3.344 · Cited by 25
“…is, it purports to describe the decision processes that are being made in generating a saccade— but that its physiological implementation is not at the level of detail found in models that show, for example, how Bayesian probabilities can be computed in a network architecture (Rao, 2004).”
BackgroundValidating model-based Bayesian integration using prior–cost metamers
Sohn, H., & Jazayeri, M. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2021531118 · Cited by 22
Cox, G. E., Palmeri, T. J., Logan, G. D., Smith, P. L., & Schall, J. D. (2022) · Psychological Review · doi:10.1037/rev0000366 · Cited by 14
“Lateral inhibition is a key element in neural systems that must deal with attenuated or dynamic input (Drugowitsch et al., 2014; Rao, 2004) and has been documented in FEF (Schall et al., 2004).”
BackgroundSpatial pooling for greyscale images
Thornton, J., & Srbic, A. (2012) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-012-0087-7 · Cited by 15
“[13,18,3,2]).”
Stable weights dynamics for a class of differential neural network observer
Chaírez, I., Poznyak, A. S., & Poznyak, T. I. (2009) · IET Control Theory and Applications · doi:10.1049/iet-cta.2008.0261 · Cited by 14
Xu, J., Yang, Z., & Tsien, J. Z. (2010) · PLoS ONE · doi:10.1371/journal.pone.0015796 · Cited by 14
“This hypothesis is conceptually distinct from the conventional view of neurons as feature detectors, the efficient coding hypothesis [10,11], predictive coding [45], the proposal that neurons encode logarithmic likelihood functions [46], and several recent V1 neuronal models that involve complex spatial-tempo Figure 7.”
BackgroundSampling-based causal inference in cue combination and its neural implementation
Yu, Z., Chen, F., Dong, J., & Dai, Q. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.10.045 · Cited by 13
Canonical cortical circuits and the duality of Bayesian inference and optimal control
Doya, K. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.07.003 · Cited by 16
“combining prior expectation and sensory evidence [18-20], hypotheses have been proposed about how Bayesian inference can be implemented in the canonical cortical circuits [21-28].”
BackgroundThe Neural Costs of Optimal Control
Gershman, S. J., & Wilson, R. C. (2010) · Cited by 14
Fluctuation-Dissipation Theorem and Models of Learning
Nemenman, I. M. (2005) · Neural Computation · doi:10.1162/0899766054322982 · Cited by 13
“To attack the problem, one can construct a biologically plausible computing machine with a known learning–theoretic equivalent (Rao, 2004) and then search for a structural similarity with a real leaving organism.”
BackgroundIntegration of form and motion within a generative model of visual cortex
Sajda, P., & Baek, K. (2004) · Neural Networks · doi:10.1016/j.neunet.2004.03.013 · Cited by 14
“A recent modeling study by Rao (2004) has shown how integrateand-fire neurons can be used to build networks for carrying out Bayesian inference.”
Implementing belief propagation in neural circuits
Shon, A. P., & Rao, R. P. N. (2004) · Neurocomputing · doi:10.1016/j.neucom.2004.10.035 · Cited by 13
Self-citationDimova, K., & Denham, M. J. (2009) · Biological Cybernetics · doi:10.1007/s00422-009-0291-z · Cited by 14
Population Codes: Theoretic Aspects
Ma, W. J., & Pouget, A. (2009) · Elsevier eBooks · doi:10.1016/b978-008045046-9.01401-7 · Cited by 12
When the Brain Meets the Eye: Tracking Object Motion
Masson, G. S., Montagnini, A., & Ilg, U. J. (2009) · Springer eBooks · doi:10.1007/978-1-4419-0781-3_8 · Cited by 13
Zheng, Y., Jia, S., Yu, Z., Huang, T., Liu, J. K., & Tian, Y. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.03.003 · Cited by 13
“The previous work of neural implementation of Bayesian inference [29, 6, 34] with recurrent neural networks focused on inference of hidden Markov models.”
BackgroundMethodNeural Models of Bayesian Belief Propagation
Rao, R. P. N. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/9780262042383.003.0011 · Cited by 18
Self-citationContextual Integration in Cortical and Convolutional Neural Networks
Iyer, R., Hu, B., & Mihalaş, Ş. (2020) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2020.00031 · Cited by 10
“…directly relate neural activity to the probability of a feature being present in the neuron’s classical RF (Barlow, 1969; Anastasio et al., 2000; Rao, 2004). in Zhu and Rozell (2015) to model inhibitory interneurons in efficient sensory coding models using matrix decomposition techniques, we…”
MethodDecoding stimulus duration from neural responses in the auditory midbrain
Aubie, B., Sayegh, R., Fremouw, T., Covey, E., & Faure, P. A. (2014) · Journal of Neurophysiology · doi:10.1152/jn.00360.2014 · Cited by 10
Motion detection and object tracking with discrete leaky integrate-and-fire neurons
Risinger, L., & Kaikhah, K. (2007) · Applied Intelligence · doi:10.1007/s10489-007-0092-9 · Cited by 10
“One such approach is presented in “Bayesian Computation in Recurrent Neural Circuits” [ 3 ].”
MethodSmoke and mirrors — Virtual realities for sensor fusion experiments in biomimetic robotics
Bauer, J., Dávila-Chacón, J., Strahl, E., & Wermter, S. (2012) · IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI) · doi:10.1109/mfi.2012.6343022 · Cited by 10
“If we understand them well enough and we can show that they are very similar in real life and in the experiment, then we can demonstrate robustness and fitness of our systems under natural conditions.”
BackgroundAn Embodied Model of Young Children’s Categorization and Word Learning
Twomey, K. E., Horst, J. S., & Morse, A. F. (2013) · IGI Global eBooks · doi:10.4018/978-1-4666-2973-8.ch008 · Cited by 9
Blind Signal Separation and Extraction: Recent Trends, Future Perspectives, and Applications
Cichocki, A. S., & Zurada, J. M. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24844-6_4 · Cited by 10
Modelling neural probabilistic computation using vector symbolic architectures
Furlong, P. M., & Eliasmith, C. (2023) · Cognitive Neurodynamics · doi:10.1007/s11571-023-10031-7 · Cited by 11
“…codes , where the firing rate is proportional to the log of the probability of the that the input is the preferred stimulus of the neuron [e.g., Rao (2004)], and log likelihood ratio codes , where neuron activity is a function of the ratio of the probability of two, binary, stimuli (Deneve…”
Spike-Based Probabilistic Inference in Analog Graphical Models Using Interspike-Interval Coding
Steimer, A., & Douglas, R. J. (2013) · Neural Computation · doi:10.1162/neco_a_00477 · Cited by 11
“Apart from ISI coding there are several other models using various spikecoding schemes that propose how belief propagation (Rao, 2004; Ott & Stoop, 2006; Steimer et al., 2009) or other methods of probabilistic inference (Yu & Dayan, 2005; Deneve, Latham, & Pouget, 2001; Deneve, 2007; Börlin & Deneve, 2011; Ma et al.”
BackgroundMethodInfluentialGlobal Expectation-Violation as Fitness Function in Evolutionary Composition
Browne, T. M., & Fox, C. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-01129-0_60 · Cited by 9
“We implemented evolutionary search using a Metropolis-Hastings (MH) algorithm with simulated annealing [19].”
MethodVisual-haptic cue integration with spatial and temporal disparity during pointing movements
Serwe, S., Kording, K. P., & Trommershäuser, J. (2011) · Experimental Brain Research · doi:10.1007/s00221-011-2603-5 · Cited by 12
“These probability distributions could, for example, be encoded directly (Beck and Pouget 2007; Rao 2004), via probabilistic population codes (Deneve et al. 2001; Ma et al. 2006) or via synaptic integration in single neurons (Deneve 2008a, b).”
BackgroundImplementing Bayes’ Rule with Neural Fields
Cuijpers, R. H., & Erlhagen, W. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87559-8_24 · Cited by 13
“This can be achieved by representing probability distributions in the log domain [7].”
BackgroundA neural network implementing optimal state estimation based on dynamic spike train decoding
Bobrowski, O., Meir, R., Shoham, S., & Eldar, Y. C. (2007) · Cited by 9
Predictive Coding in Area V4: Dynamic Shape Discrimination under Partial Occlusion
Choi, H., Pasupathy, A., & Shea‐Brown, E. (2018) · Neural Computation · doi:10.1162/neco_a_01072 · Cited by 14
“…codes, and has successfully described neural responses in a variety of different sensory systems (Bogacz, 2015; Bastos et al., 2012; Friston & Kiebel, 2009a, b; Srinivasan et al., 1982; Rao & Ballard, 1999; Spratling, 2016; Rao, 1997, 1999, 2004, 2005; Lee & Mumford, 2003; Yuille & Kersten, 2006).”
BackgroundEfficient Computation Based on Stochastic Spikes
Ernst, U. A., Rotermund, D., & Pawelzik, K. R. (2007) · Neural Computation · doi:10.1162/neco.2007.19.5.1313 · Cited by 10
“Therefore, generative models with iterative Bayesian algorithms appear to be suitable candidates for a comprehensive, abstract model for the function and dynamics of networks in the brain (Mumford, 2002; Lee & Mumford, 2003; Rao, 2004).”
MethodEmergent human-like covert attention in feedforward convolutional neural networks
Srivastava, S., Wang, W. Y., & Eckstein, M. P. (2024) · Current Biology · doi:10.1016/j.cub.2023.12.058 · Cited by 10
“It often (but not always 54 ) omits the process of learning its predictive probabilities and has a level of representation that is theoretically interpretable but more difficult to map directly to a biologically plausible large neuronal population (but see Ma et al., 30 Wei and Stocker, 55 and Rao 56 ).”
Yu, A. J. (2014) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199675111.013.025 · Cited by 9
“This spans a broad spectrum, from distributional codes that can encode mean and variance (Zemel, Dayan, & Pouget, 1998), to more exotic codes that can represent complex distributions (Sahani & Dayan, 2003; Barber, Clark, & Anderson, 2003; Rao, 2004; Weiss & Fleet, 2002).”
BackgroundMethodNeural network implementation of inference on binary Markov random fields with probability coding
Yu, Z., Chen, F., & Dong, J. (2017) · Applied Mathematics and Computation · doi:10.1016/j.amc.2016.12.025 · Cited by 8
Unification of MAP Estimation and Marginal Inference in Recurrent Neural Networks
Yu, Z., Chen, F., & Deng, F. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2805813 · Cited by 7
“Rao [22] proved that the marginal inference equation of BP for HMMs is equivalent to the dynamic equation of RNNs, which means that RNNs can implement marginal inference for HMMs directly.”
InfluentialOptimal Prediction of Moving Sound Source Direction in the Owl
Cox, W., & Fischer, B. J. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004360 · Cited by 7
“In particular, several studies have addressed the problem of inference in time in the context of hidden Markov models [20,27,28] and tracking using the Kalman filter [29,19,22,30,31].”
BackgroundA Marr's Three‐Level Analytical Framework for Neuromorphic Electronic Systems
Guo, Y., Zou, X., Hu, Y., Yang, Y., Wang, X., He, Y., Kong, R., Guo, Y., Li, G., Zhang, W., et al. (2021) · Advanced Intelligent Systems · doi:10.1002/aisy.202100054 · Cited by 16
Sampling-Tree Model: Efficient Implementation of Distributed Bayesian Inference in Neural Networks
Yu, Z., Chen, F., & Liu, J. K. (2019) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2019.2927808 · Cited by 9
“In order to simplify the multiplication of probabilities, Rao [28], [29] proposed to use the log probability code and proved that the differential equations of recurrent neural networks are in coincidence with the inference equations of the hidden Markov model, in which the computation of sum-logs was used to approxim…”
MethodHow do high-level specifications of the brain relate to variational approaches?
Viéville, T., Chemla, S., & Kornprobst, P. (2007) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2007.10.010 · Cited by 8
“This first indictment is discussed and partially answered in (Rao, 2004).”
Learning cognitive maps as structured graphs for vicarious evaluationPreprint
Rikhye, R. V., Gothoskar, N., Guntupalli, J. S., Dedieu, A., Lázaro-Gredilla, M., & George, D. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/864421 · Cited by 10
“Neurobiological circuit Like HMMs [30], cloned HMM can be readily instantiated as a neuronal circuit whose mechanistic 115”
MethodInfluentialNormalization for probabilistic inference with neurons
Eliasmith, C., & Martens, J. (2011) · Biological Cybernetics · doi:10.1007/s00422-011-0433-y · Cited by 7
“There have been several suggestions for how to implement probabilistic inference in neurons (Rao 2004; Eliasmith and Anderson 2003; Ma et al. 2006; Sahani and Dayan 2003).”
BackgroundMethodInfluentialChambers, C., Sokhey, T., Gaebler-Spira, D. J., & Kording, K. P. (2017) · PLoS ONE · doi:10.1371/journal.pone.0188741 · Cited by 10
Mechanisms of Reward-Modulated STDP and Winner-Take-All in Bayesian Spiking Decision-Making Circuit
Yan, H., Liu, X., Huo, H., & Fang, T. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-36718-3_14 · Cited by 6
Implementation of Bayesian Inference In Distributed Neural Networks
Yu, Z., Huang, T., & Liu, J. K. (2018) · Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP) · doi:10.1109/pdp2018.2018.00111 · Cited by 7
“Rao [5], [6] derived the inference equation of hidden Markov models (HMMs) with BP and demonstrated that the differential physical equation of recurrent neural circuits is consistent with the inference equation of HMMs, where a sum-logs is used to approximate a log-sum.”
MethodThe Evolutionary Origin of Nervous Systems and Implications for Neural Computation
Monk, T. (2014) · Otago University Research Archive (University of Otago) · Cited by 8
Fast decisions reflect biases; slow decisions do not
Linn, S., Lawley, S. D., Karamched, B. R., Kilpatrick, Z. P., & Josić́, K. (2024) · Physical review. E · doi:10.1103/physreve.110.024305 · Cited by 10
Revealing Structure of Complex Biological Systems Using Bayesian Networks
Smith, V. A. (2010) · Springer eBooks · doi:10.1007/978-1-84996-396-1_9 · Cited by 8
“BNs have recently gained use in neuroscience, where the needs of BNs and the available data are more congruent; additionally, the theory that neurons in fact perform Bayesian inference [ 73 ] suggests that BNs may be highly suited to modelling this biological domain.”
MethodA Computational Framework for the Study of Confidence Across Species
Kepecs, Á., & Mainen, Z. F. (2014) · Springer eBooks · doi:10.1007/978-3-642-45190-4_6 · Cited by 9
Learnability, representation, and language: a bayesian approach
Perfors, A. (2008) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 7
Bayesian model learning in human visual perception
Orbán, G., Fiser, J., Aslin, R. Ν., & Lengyel, M. (2005) · Cambridge University Engineering Department Publications Database · Cited by 6
What and where: a bayesian inference theory of visual attention
Poggio, T., & Chikkerur, S. S. (2010) · Cited by 6
The neural basis for uncertainty processing in hierarchical decision making
Wang, M. B., Lynch, N., & Halassa, M. M. (2025) · Nature Communications · doi:10.1038/s41467-025-63994-y · Cited by 13
Modeling Stereopsis via Markov Random Field
Ming, Y., & Hu, Z. (2010) · Neural Computation · doi:10.1162/neco_a_00005-ming · Cited by 6
“Rao (2004, 2005) proposed performing Bayesian inferences in the log domain, inspired by the physiological evidence for the neural encoding of log probabilities (Carpenter & Williams, 1995).”
BackgroundMethodBayesian Quantitative Electrophysiology and Its Multiple Applications in Bioengineering
Barr, R. C., Nolte, L. W., & Pollard, A. E. (2010) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2010.2089375 · Cited by 7
“Other Bayesian structural models have been presented by Jepna [81] and by Rao [82].”
MethodConstruction and evaluation of an integrated dynamical model of visual motion perception
Tlapale, É., Dosher, B. A., & Lu, Z. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.03.011 · Cited by 5
“Although general neural implementation strategies for Bayesian mechanisms have been proposed (Rao, 2004), as is the case for other optimization methods such as variational approaches (Viéville et al., 2007), the link to the effective neural computations is often not well specified.”
MethodLearning to be attractive: Probabilistic computation with dynamic attractor networks
Gepperth, A., & Lefort, M. (2016) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2016.7846831 · Cited by 6
“When comparing our approach to related work, several differences are evident: First of all, and different from [7], [10], [9], our approach does not treat neural activities as log-probabilities.”
BackgroundInfluentialSchwabe, L., & Obermayer, K. (2005) · Vision Research · doi:10.1016/j.visres.2005.05.028 · Cited by 5
“For example, in Rao and Ballard (1999) the feedback within the visual cortex can be viewed as part of a hierarchical model of the sensory data which explains the afferent input by predicting it. A related interpretation is to view sensory systems as performing Bayesian inference, where the activations represent probab…”
Bayesian Modeling of Sensory Cue Combinations
Beierholm, U. (2007) · CaltechTHESIS (California Institute of Technology) · doi:10.7907/k89b-xw75 · Cited by 5
Winner-Take-All as Basic Probabilistic Inference Unit of Neuronal CircuitsPreprint
Yu, Z., Tian, Y., Huang, T., & Liu, J. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.00675 · Cited by 6
“A few models have been proposed to relate the dynamics of spiking neural networks to the inference equations of Belief Propagation (BP) algorithm, an inference algorithm commonly used in probabilistic graphical model, which is exact on acyclic graphical models and approximate on cyclic graphical models [5, 6, 7, 8, 9,…”
BackgroundMethodProcessing and Transmission of Confidence in Recurrent Neural Hierarchies
Gepperth, A. (2013) · Neural Processing Letters · doi:10.1007/s11063-013-9311-z · Cited by 4
A motor learning neural model based on Bayesian network and reinforcement learning
Hosoya, H. (2009) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2009.5178689 · Cited by 5
“Among these, it has been shown that models based on Bayesian network can serve as a powerful and biologically plausible model and indeed have successfully explained various phenomena in visual cortex [7], [16], [22], [23].”
BackgroundNeural Predictive Computation in the Cerebellum
Tanaka, H., Ishikawa, T., & Kakei, S. (2021) · Contemporary clinical neuroscience · doi:10.1007/978-3-030-75817-2_18 · Cited by 5
Unifying perception and curiosity
Ballard, D. H., & Shaw, J. (2006) · Cited by 4
Motion detection using an aVLSI network of spiking neurons
Wang, Y., & Liu, S. (2010) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2010.5537018 · Cited by 4
“Section IV discusses the results from an aVLSI spiking network implementation of a recurrent direction-selective architecture proposed by Rao (2004) [8].”
BackgroundInfluentialKaplan, B. A., Khoei, M. A., Lansner, A., & Perrinet, L. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889847 · Cited by 4
“A model proposed by Rao et al [17] implements Bayesian inference of a hidden Markov model by interpretation of recurrent feedforward weights as log posteriors.”
MethodAttention as a Bayesian inference process
Chikkerur, S. S., Serre, T., Tan, C., & Poggio, T. (2011) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.876734 · Cited by 4
Genewein, T., & Braun, D. A. (2016) · Biological Cybernetics · doi:10.1007/s00422-016-0684-8 · Cited by 4
“One way to deal with this kind of problem is to investigate stochastic and sampling-based approximations of probabilistic update schemes [21,28,34,60,67,68].”
BackgroundIndependently Coupled HMM Switching Classifier for a Bimodel Brain-Machine Interface
Darmanjian, S., Kim, S.-P., Nechyba, M. C., Príncipe, J. C., Wessberg, J., & Nicolelis, M. (2006) · Machine learning for signal processing ... · doi:10.1109/mlsp.2006.275579 · Cited by 4
“The use of thresholds for the likelihood ratio has been used in neural science and other areas of research [20, 18].”
BackgroundMethodNeural Encoding of Scene Statistics for Surface and Object Inference
Lee, T. S., Stepleton, T., Potetz, B. R., & Samonds, J. M. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511635465.025 · Cited by 4
“This is analogous to beliefs at a node in a graphical model of a Bayes net (Rao 2004, Potetz and Lee 2008, see also Knill and Pouget 2004).”
Population approach to a neural discrimination task
Gaillard, B., Buxton, H., & Feng, J. (2005) · Biological Cybernetics · doi:10.1007/s00422-005-0039-3 · Cited by 4
“In the work of Rao (2004) the decision is made by a Bayesian network.”
MethodThe complexity dividend: when sophisticated inference mattersPreprint
Tavoni, G., Balasubramanian, V., & Gold, J. I. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/563346 · Cited by 4
“…accuracy, and complexity, leaving open basic questions about when and why each of these models is relevant to how the brain solves these problems (Rao, 2004; Bogacz et al., 2006; Gold & Shadlen, 2007; Fearnhead & Liu, 2007; Krugel et al., 2009; Shi & Griffths, 2009; Brown & Steyvers, 2009;…”
BackgroundLearning attention-controllable border-ownership for objectness inference and bindingPreprint
Dedieu, A., Rikhye, R. V., Lázaro-Gredilla, M., & George, D. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.12.31.424926 · Cited by 5
“The output of these clones is then a weighted sum of its lateral inputs, multiplied by the bottom-up input [39, 29].”
BackgroundBayesian estimation inherent in a Mexican-hat-type neural network
Takiyama, K. (2016) · Physical review. E · doi:10.1103/physreve.93.052303 · Cited by 3
Design and analysis of generative models for brain machine interfaces
Darmanjian, S. (2009) · Cited by 3
Dynamic Cue Combination in Distributional Population Code Networks
Natarajan, R. D., & Zemel, R. S. (2011) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195387247.003.0020 · Cited by 3
“model (Rao, 2004).”
Giese, M. A., Knoblich, Grosjean, M., Thornton, I. M., & Shiffrar, M. (2005) · MPG.PuRe (Max Planck Society) · Cited by 3
Modeling Cognitive Processes with Neural Reinforcement LearningPreprint
Bosch, S., Seeliger, K., & van Gerven, M. A. J. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/084111 · Cited by 3
The state of hybrid artificial intelligence for interstellar missions
Ellery, A. A. (2025) · Progress in Aerospace Sciences · doi:10.1016/j.paerosci.2025.101100 · Cited by 3
Vision as a Fundamentally Statistical Machine
Yang, Z. (2012) · InTech eBooks · doi:10.5772/50165 · Cited by 2
“…(Simoncelli & Olshausen, 2001), predictive coding (Rao & Ballard, 1999), the proposal that neurons encode logarithmic likelihood functions (Rao, 2004), and several recent V1 neuronal models that involve complex spatial-tempo structures but don't function as estimators of PDs in natural…”
A cascaded neuro-computational model for spoken word recognition
Hoya, T., & van Leeuwen, C. (2010) · Connection Science · doi:10.1080/09540090903383171 · Cited by 2
“Although Rao (2004) studies the neural-plausibility of HMM in depth, we do not dig into this issue here.”
BackgroundLatency-Based Probabilistic Information Processing in Recurrent Neural Hierarchies
Gepperth, A., & Lefort, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11179-7_90 · Cited by 4
“There exists a large body of literature[5], [4], [8], [12], [4], [6], [7] on probabilistic aspects of neural coding.”
BackgroundInfluential◾ Contour-, Surface-, and Object-Related Coding in the Visual Cortex
von der Heydt, R. (2011) · Chapman & Hall/CRC eBooks · doi:10.1201/b10518-11 · Cited by 2
Inference and segmentation in cortical processing
Liu, Y., Cecchi, G., Rao, A. R., Kozloski, J., & Peck, C. (2006) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.650804 · Cited by 2
The Brain Uses Reliability of Stimulus Information when Making Perceptual DecisionsPreprint
Bitzer, S., & Kiebel, S. J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01972 · Cited by 3
“There appears to be a general consensus that the brain implements the mechanisms required for bounded accumulation, although different models were proposed for how exactly this accumulation is employed by the brain [5, 6, 1, 7, 8].”
BackgroundResultTowards the Next Generation of Retinal Neuroprosthesis: Visual Computation with SpikesPreprint
Yu, Z., Liu, J. K., Jia, S., Zhang, Y., Zheng, Y., Tian, Y., & Huang, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.04064 · Cited by 2
“…to these coding principles, there are di ff erent ways to implement inference with a network of neurons: 1) implementing inference with neural dynamics that has equations similar to the inference equations of some probabilistic graphical models over the time course [126, 127, 129, 139, 140, 141].”
BackgroundMethodAn investigation of fast and slow mapping
Twomey, K. E. (2012) · Sussex Research Online (University of Sussex) · Cited by 2
Latency-based probabilistic information processing in a learning feedback hierarchy
Gepperth, A. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889919 · Cited by 1
“There exists a large body of literature[5], [4], [8], [12], [4], [6], [7] on probabilistic aspects of neural coding.”
BackgroundInfluential
Metadata
- Authors
- Rao, R. P. N.
- Affiliations
- University of Washington (US)
- Corresponding author
- Rao, Rajesh P. N.
- ORCID
- Rao, Rajesh P. N.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2004-01-01
- Volume
- 16
- Issue
- 1
- Pages
- 1-38
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2133861842
- Cited by
- 252
- References
- 78
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 5 · 2023: 8 · 2022: 5 · 2021: 8 · 2020: 8 · 2019: 5 · 2018: 12 · 2017: 8 · 2016: 10 · 2015: 12 · 2014: 20 · 2013: 10 · 2012: 16
- FWCI
- 9.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- Bayesian inference, recurrent neural circuits, hidden Markov models, orientation discrimination, visual motion detection, random dot motion, direction selectivity, posterior probability estimation, evidence accumulation, lateral intraparietal area, frontal eye field
- MeSH terms
- Action Potentials, Animals, Bayes Theorem, Cerebral Cortex, Discrimination Learning, Humans, Markov Chains, Motion Perception, Nerve Net, Neural Pathways, Synaptic Transmission, Neurons, Orientation, Pattern Recognition, Visual, Photic Stimulation, Reaction Time, Visual Fields, Artifacts, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.