Bayesian Computation in Recurrent Neural Circuits
The DOI matches its Crossref record; 77 of the 78 references it cites were found in a registry or an index.
Abstract
A large number of human psychophysical results have been successfully explained in recent years using Bayesian models. However, the neural implementation of such models remains largely unclear. In this article, we show that a network architecture commonly used to model the cerebral cortex can implement Bayesian inference for an arbitrary hidden Markov model. We illustrate the approach using an orientation discrimination task and a visual motion detection task. In the case of orientation discrimination, we show that the model network can infer the posterior distribution over orientations and correctly estimate stimulus orientation in the presence of significant noise. In the case of motion detection, we show that the resulting model network exhibits direction selectivity and correctly computes the posterior probabilities over motion direction and position. When used to solve the well-known random dots motion discrimination task, the model generates responses that mimic the activities of evidence-accumulating neurons in cortical areas LIP and FEF. The framework we introduce posits a new interpretation of cortical activities in terms of log posterior probabilities of stimuli occurring in the natural world.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Message-passing inference in CSCG is computationally cheap and biologically plausible using simple integrate-and-fire neurons (23).”
From Space is a latent sequence: A theory of the hippocampus (2024) · cites this paper as [23]
In the citing paper
“Moreover, there are population coding mechanisms that could support explicit probabilistic computations (Zemel and Dayan, 1997; Sahani and Dayan, 2003; Rao, 2004; Ma et al., 2006; Eliasmith and Martens, 2011; Gershman and Beck, 2016).”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Rao, 2004
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444 · cited 1×
“We intend to explore the question of synaptic learning using biologically-plausible approximations to the expectation-maximization (EM) algorithm (Dempster, Laird, & Rubin, 1977).”
MethodHebb, D. O. (2005) · doi:10.4324/9781410612403 · Cited by 9,825
Distributed hierarchical processing in the primate cerebral cortex
Felleman, D. J., & Van Essen, D. C. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1 · Cited by 7,783 · cited 2×
“Such models would provide a better approximation to the multi-layered architecture of the cortex and the hierarchical connections that exist between cortical areas (Felleman & Van Essen, 1991).”
BackgroundRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 4×
“…question is how the suggested framework could be extended to allow neural implementation of hierarchical Bayesian inference in graphical models that are more general than hidden Markov models (cf. (Dayan et al., 1995; Dayan & Hinton, 1996; Hinton & Ghahramani, 1997; Rao & Ballard, 1997, 1999)).”
BackgroundMethodResultInfluentialSelf-citationAn introduction to hidden Markov models
Rabiner, L., & Juang, B. (1986) · IEEE ASSP Magazine · doi:10.1109/massp.1986.1165342 · Cited by 4,775 · cited 1×
“For standard hidden Markov models, the E-step in the EM algorithm is realized by the “forward-backward” algorithm (Rabiner & Juang, 1986).”
MethodTheoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Dayan, P., & Abbott, L. F. (2001) · Cited by 4,099
The Handbook of brain theory and neural networks
(1996) · Choice Reviews Online · doi:10.5860/choice.33-4484 · Cited by 3,870
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588
Parallel & distributed processing
Slusallek, P., Shirley, P., Mark, W., Stoll, G., & Wald, I. (2005) · ACM SIGGRAPH 2005 Courses on - SIGGRAPH '05 · doi:10.1145/1198555.1198750 · Cited by 2,568
Biophysics of Computation: Information Processing in Single Neurons
Koch, C. (1998) · Oxford University Press eBooks · doi:10.1093/oso/9780195104912.001.0001 · Cited by 2,283
Research works printed but not linked: 1
- 10.1037/0033-295X.106.2.261 · doi:10.1037/0033-295x.106.2.261 · Paper page
Cited by
Items 201–251 of 251
Estimation of State Transition Probabilities: A Neural Network Model
Saitō, H., Takiyama, K., & Okada, M. (2015) · Journal of the Physical Society of Japan · doi:10.7566/jpsj.84.124801 · Cited by 1
Spiking, Salience, and Saccades: Using Cognitive Models to Bridge the Gap Between “How” and “Why”
Cox, G. E., Palmeri, T. J., Logan, G. D., Smith, P. L., & Schall, J. D. (2023) · Springer eBooks · doi:10.1007/978-3-031-45271-0_6 · Cited by 1
Furlong, P. M., Simone, K., Dumont, N. S.-Y., Bartlett, M. E., Stewart, T. C., Orchard, J., & Eliasmith, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72341-4_7 · Cited by 1
“Prior approaches to capturing probability in neural systems have proposed methods that map random variables onto individual neurons ( e.g. , 32, 31, 7, 27, 4).”
MethodProbabilistic brains: knowns and
Beck, J. M., Ma, W. J., & Latham, P. E. (2013) · Cited by 1
The Development of Brain Theory
Palm, G. (2022) · Springer eBooks · doi:10.1007/978-3-031-00311-0_10 · Cited by 1
How is sensory information processed
Nemenman, I. M. (2004) · Cited by 1
Dawson, C. (2011) · UA Campus Repository (The University of Arizona) · Cited by 1
Inferring direction of figure using a recurrent integrate-and-fire neural circuit
Baek, K., Kim, D., & Sajda, P. (2005) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2004.1404269 · Cited by 1
“Comparing equation (1) and equation (2), it has been shown that the recurrent integrate-and-fire network computation can implement Bayesian inference [9].”
BackgroundMethodInfluentialDarmanjian, S., Paiva, A. R. C., Príncipe, J. C., & Sanchez, J. C. (2007) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/ijcnn.2007.4371449
“The use of thresholds for the likelihood ratio has been used in neural science and other areas of research [7], [11].”
MethodModeling human reaching and grasping: cortex, rehabilitation and lateralization
Han, C. E. (2010) · University of Southern California Digital Library · doi:10.25549/usctheses-m2603 · Cited by 1
Configurable analog hardware for neuromorphic Bayesian inference and least-squares solutions
Shapero, S. (2013) · SMARTech Repository (Georgia Institute of Technology) · Cited by 2
Ach and NE : Bayes, uncertainty, attention and learning.
Yu, A. J. (2005) · TUbilio (Technical University of Darmstadt) · Cited by 1
How the brain might work: statistics flowing in redundant population codesPreprint
Pitkow, X., & Angelaki, D. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.03492 · Cited by 1
“In spatial representations of probability, the encoded probability distribution is determined by the spatial pattern of which neurons are active (Ma et al. 2006; Jazayeri and Movshon 2006; Savin and Denève 2014; Rao 2004).”
BackgroundCox, G. E., Palmeri, T. J., Logan, G. D., Smith, P. L., & Schall, J. D. (2021) · doi:10.31234/osf.io/rkh8g · Cited by 1
Srivastava, S., Wang, W. Y., & Eckstein, M. P. (2022) · Research Square · doi:10.21203/rs.3.rs-1805213/v1 · Cited by 1
Chakraborty, A., & Chakrabarti, S. (2023) · doi:10.36227/techrxiv.22695691.v1 · Cited by 1
Is the brain a Bayesian machine
Rustichini, A. (2005) · Acta Neurobiologiae Experimentalis
MSE-based analysis of optimal tuning functions predicts phenomena observed in sensory neuronsPreprint
Yaeli, S., & Meir, R. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1002.2251
“ical results can be explained by Bayesian models, suggesting that organisms may indeed be employing Bayesian calculations in their decision-making (Knill and Richards (1996); Knill and Pouget (2004); Rao (2004); Ma et al. (2006)). Optimal real-time decoding methods have been proposed over the past decade for static (Z…”
BackgroundBLIND EXTRACTION OF SPARSE IMAGES FROM UNDER-DETERMINED MIXTURES
Kasprzak, W., Cichocki, A. S., & Okazaki, A. F. (2006) · Kluwer Academic Publishers eBooks · doi:10.1007/1-4020-4179-9_120
A Bayesian Algorithm for Motion and Structure Estimation from Image Sequences
Wu, S., & von der Malsburg, C. (2007) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/ijcnn.2007.4371430
“A possible artificial neural network implementation has been proposed using population coding and recurrent neural networks ([11], [12]).”
MethodShape representation in primate visual area 4 and inferotemporal cortex
Litt, B., & Murphy, T. M. (2010) · Scholarly Commons (University of Pennsylvania)
Vision as an anticipatory process
Hamker, F. H., & Psychologie, A. (2004)
Electrical Engineering (Intelligent Systems, Robotics, and Control)
Wipf, D., Elkan, C., Kreutz-Delgado, K., B, T. J. S. A., & Vasconcelos, N. M. (2006)
Neural learning of Kalman filtering, Kalman control, and system identification
Linsker, R. (2009) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2009.5178934
“The last of these approaches has made use of Bayesian inference and generative models, and various NN methods are motivated by, approximate, or perform a portion of the Bayesian inference process[17-27].”
MethodA Biologically Realizable Bayesian Computation in a Cortical Neural Network
Futagi, D., & Kitano, K. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33269-2_32
Futagi, D., & Kitano, K. (2016) · Nonlinear Theory and Its Applications IEICE · doi:10.1587/nolta.7.146
“In the previous study [4], Bayesian estimation was implemented by a recurrent network that consists of neurons with firing-rate dynamics.”
MethodInfluentialA biologically inspired model of decision making controlled by random neural activity.
Gaillard, B. (2007) · OpenGrey (Institut de l'Information Scientifique et Technique)
Sejnowski, B. J., Howard Poizner, A., LYNCH, G. S., Gepshtein, S., & Greenspan, R. J. (2014)
Neural Representation, Learning and Manipulation of Uncertainty
Natarajan, R. D. (2010) · TSpace (University of Toronto)
Neuronale Kodierung und Dekodierung von multiplen und dynamischen Reizen
Schulzke, E. L. (2006) · State and University Library Bremen
Tang, G. (2017) · doi:10.7282/t3z03c3t
“2 can be implemented using spiking neural network if it is performed in the logarithm domain [35].”
MethodComputational models of belief propagation
Ng, K. H. (2015) · doi:10.25560/55283
Hidden State Inference in the Midbrain Dopamine System
Starkweather, C. K. (2018) · Digital Access to Scholarship at Harvard (DASH) (Harvard University)
Srivastava, S., Wang, W. Y., & Eckstein, M. P. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/tymfn · Cited by 1
Metamers of Bayesian computationPreprint
Sohn, H., & Jazayeri, M. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.08.11.246355
◾ 3D and Spatiotemporal Interpolation in Object and Surface Formation
(2011) · Chapman & Hall/CRC eBooks · doi:10.1201/b10518-13
Chakraborty, A., & Chakrabarti, S. (2023) · doi:10.36227/techrxiv.22695691
A theory of inference and learning in cortex with spiking neurons and dendritic error computation
Mikulasch, F. A. (2023) · eDiss (Georg-August-Universität Göttingen) · doi:10.53846/goediss-10168 · Cited by 1
Predictive Coding in Cortical and Corticothalamic Hierarchical Networks
Rao, R. P. N., Jiang, L. P., & Fisher, A. (2023) · Oxford University Press eBooks · doi:10.1093/med/9780197676158.003.0064 · Cited by 1
Self-citationChakraborty, A., & Chakrabarti, S. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3502472
“Various forms of artificial neural networks (ANN) have been proposed by researchers for probabilistic inference of HMMs. Rao [5] has suggested that the dynamic process of a recurrent neural network (RNN) is equivalent to probabilistic Bayesian inference of an HMM.”
Canonical cortical circuits: A unified sampling machine for static and dynamic inferencePreprint
Sale, E., & Zhang, W. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.07.06.663395
“(7) Although the RBF with Gaussian case exists exact inference via Kalman filter, the circuit implementation of Kalman filter is not straightforward (Rao, 2004; Beck & Pouget, 2007) (see more in Discussion).”
InfluentialA Computational Model of Action Specification in the Basal GangliaPreprint
Bartlett, M. E., Furlong, P. M., Stewart, T. C., & Orchard, J. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.08.12.669938 · Cited by 1
“For example, this could be achieved by using methods such as log probability code [ e.g. , 69], probability ratio codes [70], explicit probability codes [ e.g. , 71, 72], convolutional probability codes [73–77], probabilistic population codes (PPCs) [78–82], or neural sampling methods [83–86].”
Hemifield-Specific Motion Extrapolation Reveals Limits of Interhemispheric Integration
Olenick, C. E., & Fallah, M. (2026) · eNeuro · doi:10.1523/eneuro.0004-26.2026
“Internal models of motion extrapolation are formalized as priors that are updated by recent motion and improved by coherent motion (Rao, 2004).”
Spiking neural networks with uncertainty model of stochastic sampling for circuit yield enhancement
Huang, Z., Xiao, W., Ruan, H., He, S., & Guo, D. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.112523
Information-Theoretic Measures of Metacognition: Bounds and Relation to Group Performance
Meyen, S., Göppert, F., Schrenk, C., von Luxburg, U., & Franz, V. H. (2025) · Open Mind · doi:10.1162/opmi.a.40 · Cited by 5
“In combination, these three results hint towards an inherent log odds representation of confidence ratings in humans, possibly due to the advantages of their additivity (Dayan & Daw, 2008; Rao, 2004).”
Rao, R. P. N. (2009) · Elsevier eBooks · doi:10.1016/b978-008045046-9.01399-1 · Cited by 1
Self-citationComputational learning noise in human decision-making
Findling, C. (2018) · theses.fr (ABES) · doi:10.70675/32e62279zea97z45b1z9326z06a449dae472
Low-rank network models of neural computations
Valente, A. (2022) · theses.fr (ABES) · doi:10.70675/51cf5234z10f7z42b2z8d46z13894170c780
Role of goal-orientated attention and expectations in visual processing and perception
Chalk, M. (2013) · ERA · Cited by 1
Metadata
- Authors
- Rao, R. P. N.
- Affiliations
- University of Washington (US)
- Corresponding author
- Rao, Rajesh P. N.
- ORCID
- Rao, Rajesh P. N.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2004-01-01
- Volume
- 16
- Issue
- 1
- Pages
- 1-38
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2133861842
- Cited by
- 252
- References
- 78
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 5 · 2023: 8 · 2022: 5 · 2021: 8 · 2020: 8 · 2019: 5 · 2018: 12 · 2017: 8 · 2016: 10 · 2015: 12 · 2014: 20 · 2013: 10 · 2012: 16
- FWCI
- 9.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- Bayesian inference, recurrent neural circuits, hidden Markov models, orientation discrimination, visual motion detection, random dot motion, direction selectivity, posterior probability estimation, evidence accumulation, lateral intraparietal area, frontal eye field
- MeSH terms
- Action Potentials, Animals, Bayes Theorem, Cerebral Cortex, Discrimination Learning, Humans, Markov Chains, Motion Perception, Nerve Net, Neural Pathways, Synaptic Transmission, Neurons, Orientation, Pattern Recognition, Visual, Photic Stimulation, Reaction Time, Visual Fields, Artifacts, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.