Bee foraging in uncertain environments using predictive hebbian learning
Abstract
Recent work has identified a neuron with widespread projections to odour processing regions of the honeybee brain whose activity represents the reward value of gustatory stimuli. We have constructed a model of bee foraging in uncertain environments based on this type of neuron and a predictive form of hebbian synaptic plasticity. The model uses visual input from a simulated three-dimensional world and accounts for a wide range of experiments on bee learning during foraging, including risk aversion. The predictive model shows how neuromodulatory influences can be used to bias actions and control synaptic plasticity in a way that goes beyond standard correlational mechanisms. Although several behavioural models of conditioning in bees have been proposed, this model is based on the neural substrate and was tested in a simulation of bee flight.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The perspective on optimum policies as prior expectations may call for a reappraisal of reinforcement learning as an explanation for behavior and the role of dopamine in coding unpredicted reward.
“The perspective on optimum policies as prior expectations may call for a reappraisal of reinforcement learning as an explanation for behavior and, in particular, the role of dopamine in coding unpredicted reward (Montague et al. 1995;Schultz et al. 1997;Doya 2002)”
From Action and behavior: a free-energy formulation (2010) · cites this paper as Montague et al. 1995
Cites
Hebb, D. O. (2005) · doi:10.4324/9781410612403 · Cited by 9,825 · see citing works
Animal choice behavior and the evolution of cognitive architecture
Real, L. A. (1991) · Science · doi:10.1126/science.1887231 · Cited by 377 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Self-citationThe free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · see citing works
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564 · see citing works
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574 · see citing works
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Self-citationLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053 · see citing works
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779 · see citing works
Self-citationThe free-energy principle: a rough guide to the brain?
Friston, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.04.005 · Cited by 1,831 · see citing works
Optimality principles in sensorimotor control
Todorov, E. (2004) · Nature Neuroscience · doi:10.1038/nn1309 · Cited by 1,746 · see citing works
A free energy principle for the brain
Friston, K., Kilner, J., & Harrison, L. (2006) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2006.10.001 · Cited by 1,369 · see citing works
Schultz, W. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.007 · Cited by 1,028 · see citing works
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955 · see citing works
Hierarchical Models in the Brain
Friston, K. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000211 · Cited by 952 · see citing works
“have been implicated in value-learning [68–70], attention and the encoding of uncertainty [63,71].”
BackgroundAction and behavior: a free-energy formulation
Friston, K. J., Daunizeau, J., Kilner, J., & Kiebel, S. J. (2010) · Biological Cybernetics · doi:10.1007/s00422-010-0364-z · Cited by 866 · see citing works
“The perspective on optimum policies as prior expectations may call for a reappraisal of reinforcement learning as an explanation for behavior and, in particular, the role of dopamine in coding unpredicted reward (Montague et al. 1995; Schultz et al. 1997; Doya 2002): If dopamine encodes precision through its classical…”
BackgroundMontague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.11.018 · Cited by 809 · see citing works
Self-citationPredictability modulates human brain response to reward
Berns, G. S., McClure, S. M., Pagnoni, G., & Montague, P. R. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-08-02793.2001 · Cited by 711 · see citing works
Self-citationChoosing the greater of two goods: neural currencies for valuation and decision making
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2005) · Nature Reviews Neuroscience · doi:10.1038/nrn1666 · Cited by 536 · see citing works
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500 · see citing works
Writing Memories with Light-Addressable Reinforcement Circuitry
Claridge-Chang, A., Roorda, R. D., Vrontou, E., Sjulson, L., Li, H., Hirsh, J., & Miesenböck, G. (2009) · Cell · doi:10.1016/j.cell.2009.08.034 · Cited by 484 · see citing works
Reinforcement learning or active inference?
Friston, K. J., Daunizeau, J., & Kiebel, S. J. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006421 · Cited by 443 · see citing works
Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., & Platt, M. L. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.4652-10.2011 · Cited by 379 · see citing works
Serotonin in Affective Control
Dayan, P., & Huys, Q. J. M. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135607 · Cited by 381 · see citing works
Self-citationRedish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.784 · Cited by 346 · see citing works
Waddell, S., Armstrong, J. D., Kitamoto, T., Kaiser, K., & Quinn, W. G. (2000) · Cell · doi:10.1016/s0092-8674(00)00183-5 · Cited by 332 · see citing works
Active inference and agency: optimal control without cost functions
Friston, K., Samothrakis, S., & Montague, R. (2012) · Biological Cybernetics · doi:10.1007/s00422-012-0512-8 · Cited by 241 · see citing works
Computational Models of Performance Monitoring and Cognitive Control
Alexander, W. H., & Brown, J. W. (2010) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2010.01085.x · Cited by 193 · see citing works
Towards a neuronal gauge theory
Sengupta, B., Tozzi, A., Cooray, G. K., Douglas, P. K., & Friston, K. J. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002400 · Cited by 132 · see citing works
Computational and dynamic models in neuroimaging
Friston, K. J., & Dolan, R. J. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.068 · Cited by 131 · see citing works
Changeux, J.-P. (2006) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2006.1832 · Cited by 31 · see citing works
He, J. M., Huang, X. T., Yuan, H., & Chen, Y. G. (2012) · Consciousness and Cognition · doi:10.1016/j.concog.2012.08.006 · Cited by 13 · see citing works
- Authors
- Montague, P. R., Dayan, P., Person, C., & Sejnowski, T. J.
- Affiliations
- Baylor College of Medicine (US)
- Massachusetts Institute of Technology (US)
- Venue
- Nature
- Published
- 1995-10
- Volume
- 377
- Issue
- 6551
- Pages
- 725-728
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2005866069
- Cited by
- 311
- References
- 2
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 6 · 2023: 10 · 2022: 18 · 2021: 7 · 2020: 7 · 2019: 7 · 2018: 5 · 2017: 4 · 2016: 7 · 2015: 13 · 2014: 8 · 2013: 11 · 2012: 16
- FWCI
- 4.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurobiology and Insect Physiology Research
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- honey bee foraging, risk aversion, neuromodulation, synaptic plasticity, visual navigation, insect learning
- MeSH terms
- Animals, Bees, Computer Simulation, Environment, Feeding Behavior, Learning, Models, Neurological, Neurons
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
