Goal-Directed Decision Making with Spiking Neurons
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
Behavioral and neuroscientific data on reward-based decision making point to a fundamental distinction between habitual and goal-directed action selection. The formation of habits, which requires simple updating of cached values, has been studied in great detail, and the reward prediction error theory of dopamine function has enjoyed prominent success in accounting for its neural bases. In contrast, the neural circuit mechanisms of goal-directed decision making, requiring extended iterative computations to estimate values online, are still unknown. Here we present a spiking neural network that provably solves the difficult online value estimation problem underlying goal-directed decision making in a near-optimal way and reproduces behavioral as well as neurophysiological experimental data on tasks ranging from simple binary choice to sequential decision making. Our model uses local plasticity rules to learn the synaptic weights of a simple neural network to achieve optimal performance and solves one-step decision-making tasks, commonly considered in neuroeconomics, as well as more challenging sequential decision-making tasks within 1 s. These decision times, and their parametric dependence on task parameters, as well as the final choice probabilities match behavioral data, whereas the evolution of neural activities in the network closely mimics neural responses recorded in frontal cortices during the execution of such tasks. Our theory provides a principled framework to understand the neural underpinning of goal-directed decision making and makes novel predictions for sequential decision-making tasks with multiple rewards. SIGNIFICANCE STATEMENT Goal-directed actions requiring prospective planning pervade decision making, but their circuit-level mechanisms remain elusive. We show how a model circuit of biologically realistic spiking neurons can solve this computationally challenging problem in a novel way. The synaptic weights of our network can be learned using local plasticity rules such that its dynamics devise a near-optimal plan of action. By systematically comparing our model results to experimental data, we show that it reproduces behavioral decision times and choice probabilities as well as neural responses in a rich set of tasks. Our results thus offer the first biologically realistic account for complex goal-directed decision making at a computational, algorithmic, and implementational level.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 1×
“Research in animal learning and behavioral neuroscience has given rise to the view that reward-based decision making is governed by (at least) two distinct strategies: a habit system, which relies on cached associations between actions or situations and their long-run future values, and a goal-directed system, which i…”
BackgroundThe time course of perceptual choice: The leaky, competing accumulator model.
Usher, M., & McClelland, J. L. (2001) · Psychological Review · doi:10.1037/0033-295x.108.3.550 · Cited by 2,152 · cited 1×
“Previous models relied on stochasticity in the inputs to the network, even for non-stochastic stimuli, and the fact that this stochasticity needs to be integrated out over time (Usher and McClelland, 2001; Smith and Ratcliff, 2004).”
MethodDissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 1×
“In particular, temporal-difference algorithms of model-free learning account for both behavioral and neuroimaging data regarding habitbased decision making (O’Doherty et al., 2004; Seymour et al., 2004).”
MethodFunctional role of the supplementary and pre-supplementary motor areas
Nachev, P., Kennard, C., & Husain, M. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2478 · Cited by 1,928
Reverse replay of behavioural sequences in hippocampal place cells during the awake state
Foster, D. J., & Wilson, M. A. (2006) · Nature · doi:10.1038/nature04587 · Cited by 1,727 · cited 2×
“Reverse replay has only been observed in the hippocampus (Foster and Wilson, 2006; Diba and Buzsa´ki, 2007), and it remains to be tested whether, for example, it also exists in PFC or whether there are anti-Hebbian forms of plasticity operating there.”
BackgroundThe importance of mixed selectivity in complex cognitive tasks
Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013) · Nature · doi:10.1038/nature12160 · Cited by 1,639 · cited 1×
“We consider neurons with mixed selectivity, a phenomenon widely observed in prefrontal cortex (PFC) and important for solving complex cognitive tasks (Rigotti et al., 2013).”
BackgroundNeurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598 · cited 10×
“Theoveralltimecoursesof neuronsweresimilartothosefoundinOFC,withonekeydiffer-ence:neuronscodingfortheunchosenoptionBdidnotshowan initialincreaseinactivity(Fig.3 C , purple lines).”
BackgroundResultInfluentialSelf-improving reactive agents based on reinforcement learning, planning and teaching
Lin, L.-J. (1992) · Machine Learning · doi:10.1007/bf00992699 · Cited by 1,564 · cited 1×
“…goal-directeddecisionmaking,suchastheventralstriatum(Pen-nartzetal.,2004;Lansinketal.,2008),thePFC(Eustonetal., 2007;Peyracheetal.,2009),andthehippocampus(Fosterand Wilson,2006;Guptaetal.,2010),mayalsoservetoreinforceand consolidateinternalmodelsoftheenvironment(Lin,1992;Mnih etal.,2015).”
Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition
Gerstner, W., Kistler, W. M., Naud, R., & Paninski, L. (2014) · doi:10.1017/cbo9781107447615 · Cited by 1,515 · cited 2×
“For biological plausibility, we used a canonical, phenomenological single-neuron model, the spike–response model (Gerstner et al., 2014), or, equivalently, the generalized linear model (Pillow et al., 2008; which has been demonstrated to be an accurate predictor of neural responses in a wide…”
MethodThe Complexity of Markov Decision Processes
Papadimitriou, C. H., & Tsitsiklis, J. N. (1987) · Mathematics of Operations Research · doi:10.1287/moor.12.3.441 · Cited by 1,329 · cited 1×
“Thus, there is a strongly nonlinear, recursive relationship between the optimal values of different actions (formalized by the so-called Bellman optimality equation; Bellman, 1957), which makes the computation of optimal values challenging (Papadimitriou and Tsitsiklis, 1987).”
Background
Cited by
Learning task-state representations
Niv, Y. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0470-8 · Cited by 356
With or without you: predictive coding and Bayesian inference in the brain
Aitchison, L., & Lengyel, M. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.08.010 · Cited by 308
“Similarly, the computational goal of reinforcement learning can be achieved by several algorithms that do not compute and represent prediction errors per se [51, 52], and whether prediction error-based or these other algorithms should be used depends on environmental and neural constraints [53].”
BackgroundSelf-citationOrbitofrontal Cortex: A Neural Circuit for Economic Decisions
Padoa-Schioppa, C., & Conen, K. E. (2017) · Neuron · doi:10.1016/j.neuron.2017.09.031 · Cited by 306
A review of inflammatory mechanism in airway diseases
Aghasafari, P., George, U., & Pidaparti, R. (2019) · Inflammation Research · doi:10.1007/s00011-018-1191-2 · Cited by 300
Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits
Payeur, A., Guerguiev, J., Zenke, F., Richards, B. A., & Naud, R. (2021) · Nature Neuroscience · doi:10.1038/s41593-021-00857-x · Cited by 292
Metadata
- Authors
- Friedrich, J., & Lengyel, M.
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2016-02-03
- Volume
- 36
- Issue
- 5
- Pages
- 1529-1546
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- OpenAlex
- W2255822949
- Cited by
- 57
- References
- 31
- FWCI
- 4.70
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Goal-Directed Decision Making with Spiking Neurons. https://app.cochat.ai/papers/10.1523/jneurosci.2854-15.2016
- The paper
- Friedrich, J., & Lengyel, M. (2016). Goal-Directed Decision Making with Spiking Neurons. The Journal of Neuroscience. https://doi.org/10.1523/jneurosci.2854-15.2016
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.