Locus Coeruleus tracking of prediction errors optimises cognitive flexibility: An Active Inference model
Open accessPublished version of a preprint
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
The locus coeruleus (LC) in the pons is the major source of noradrenaline (NA) in the brain. Two modes of LC firing have been associated with distinct cognitive states: changes in tonic rates of firing are correlated with global levels of arousal and behavioural flexibility, whilst phasic LC responses are evoked by salient stimuli. Here, we unify these two modes of firing by modelling the response of the LC as a correlate of a prediction error when inferring states for action planning under Active Inference (AI). We simulate a classic Go/No-go reward learning task and a three-arm foraging task and show that, if LC activity is considered to reflect the magnitude of high level ‘state-action’ prediction errors, then both tonic and phasic modes of firing are emergent features of belief updating. We also demonstrate that when contingencies change, AI agents can update their internal models more quickly by feeding back this state-action prediction error – reflected in LC firing and noradrenaline release – to optimise learning rate, enabling large adjustments over short timescales. We propose that such prediction errors are mediated by cortico-LC connections, whilst ascending input from LC to cortex modulates belief updating in anterior cingulate cortex (ACC). In short, we characterise the LC/ NA system within a general theory of brain function. In doing so, we show that contrasting, behaviour-dependent firing patterns are an emergent property of the LC’s crucial role in translating prediction errors into an optimal mediation between plasticity and stability. Author Summary The brain uses sensory information to build internal models and make predictions about the world. When errors of prediction occur, models must be updated to ensure desired outcomes are still achieved. Neuromodulator chemicals provide a possible pathway for triggering such changes in brain state. One such neuromodulator, noradrenaline, originates predominantly from a cluster of neurons in the brainstem – the locus coeruleus (LC) – and plays a key role in behaviour, for instance, in determining the balance between exploiting or exploring the environment. Here we use Active Inference (AI), a mathematical model of perception and action, to formally describe LC function. We propose that LC activity is triggered by errors in prediction and that the subsequent release of noradrenaline alters the rate of learning about the environment. Biologically, this describes an LC-cortex feedback loop promoting behavioural flexibility in times of uncertainty. We model LC output as a simulated animal performs two tasks known to elicit archetypal responses. We find that experimentally observed ‘phasic’ and ‘tonic’ patterns of LC activity emerge naturally, and that modulation of learning rates improves task performance. This provides a simple, unified computational account of noradrenergic computational function within a general model of behaviour.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
Berridge, C. W., & Waterhouse, B. D. (2003) · Brain Research Reviews · doi:10.1016/s0165-0173(03)00143-7 · Cited by 2,529 · cited 7×
“They may also include stimuli that are highly 67 unexpected (1,11) – although the phasic response will habituate rapidly to novelty alone in the 68 absence of behavioural salience (12).”
BackgroundInfluentialLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Aston-Jones, G., & Bloom, F. (1981) · The Journal of Neuroscience · doi:10.1523/jneurosci.01-08-00876.1981 · Cited by 1,960 · cited 1×
“They may also include stimuli that are highly 67 unexpected (1,11) – although the phasic response will habituate rapidly to novelty alone in the 68 absence of behavioural salience (12).”
BackgroundNucleus locus ceruleus: new evidence of anatomical and physiological specificity
Foote, S. L., Bloom, F. E., & Aston-Jones, G. (1983) · Physiological Reviews · doi:10.1152/physrev.1983.63.3.844 · Cited by 1,901
Decision making, the P3, and the locus coeruleus–norepinephrine system
Nieuwenhuis, S., Aston-Jones, G., & Cohen, J. D. (2005) · Psychological Bulletin · doi:10.1037/0033-2909.131.4.510 · Cited by 1,678
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Active inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156 · cited 6×
“Formally, these describe the path integral of free energy expected under competing policies (see [14] and Appendix 1).”
BackgroundMethodInfluentialSelf-citationOrienting and reorienting: the locus coeruleus mediates cognition through arousal
Sara, S. J., & Bouret, S. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.011 · Cited by 1,056
Choice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866
Research works printed but not linked: 26
- DGR Tervo, 2014, Behavioral variability through stochastic choice and its gating by anterior cingulate cortex, Cell, 159, 21, 10.1016/j.cell.2014.08.037 · DOI · Paper page
- AJ Yu, 2003, Expected and unexpected uncertainty: ACh and NE in the neocortex, Adv neural Inf Process, 15, 157
- G Aston-Jones, 1981, Norepinephrine-containing locus coeruleus neurons in behaving rats exhibit pronounced responses to non-noxious environmental stimuli, J Neurosci, 1, 887, 10.1523/JNEUROSCI.01-08-00887.1981 · DOI · Paper page
- S Bouret, 2015, Sensitivity of Locus Ceruleus Neurons to Reward Value for Goal-Directed Actions, J Neurosci, 35, 4005, 10.1523/JNEUROSCI.4553-14.2015 · DOI · Paper page
- P Dayan, 2006, Phasic norepinephrine: A neural interrupt signal for unexpected events, Netw Comput Neural Syst December, 17, 335, 10.1080/09548980601004024 · DOI · Paper page
- S Bouret, 2005, Network reset: A simplified overarching theory of locus coeruleus noradrenaline function, Trends in Neurosciences, 28, 574, 10.1016/j.tins.2005.09.002 · DOI · Paper page
- S Bouret, 2004, Reward expectation, orientation of attention and locus coeruleus-medial frontal cortex interplay during learning, Eur J Neurosci, 20, 791, 10.1111/j.1460-9568.2004.03526.x · DOI · Paper page
- P Schwartenbeck, 2013, Exploration, novelty, surprise, and free energy minimization, Front Psychol, 4, 710, 10.3389/fpsyg.2013.00710 · DOI · Paper page
- G Aston-Jones, 1997, Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task, Neuroscience, 80, 697, 10.1016/S0306-4522(97)00060-2 · DOI · Paper page
- MS Gilzenrat, 2010, Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function, Cogn Affect Behav Neurosci, 10, 252, 10.3758/CABN.10.2.252 · DOI · Paper page
- M Jepma, 2011, Pupil Diameter Predicts Changes in the Exploration?Exploitation Trade-off: Evidence for the Adaptive Gain Theory, J Cogn Neurosci, 23, 1587, 10.1162/jocn.2010.21548 · DOI · Paper page
- K Preuschoff, 2011, Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making, Front Neurosci, 5, 115, 10.3389/fnins.2011.00115 · DOI · Paper page
- RM Kalwani, 2014, Phasic activation of individual neurons in the locus ceruleus/subceruleus complex of monkeys reflects rewarded decisions to go but not stop, J Neurosci, 34, 13656, 10.1523/JNEUROSCI.2566-14.2014 · DOI
- M Matsumoto, 2007, Medial prefrontal cell activity signaling prediction errors of action values, Nat Neurosci, 10, 647, 10.1038/nn1890 · DOI · Paper page
- BY Hayden, 2011, Neuronal basis of sequential foraging decisions in a patchy environment, Nat Neurosci, 14, 933, 10.1038/nn.2856 · DOI · Paper page
- MP Karlsson, 2012, Network Resets in Medial Prefrontal Cortex Mark the Onset of Behavioral Uncertainty, Science, 338, 135, 10.1126/science.1226518 · DOI · Paper page
- RB Ebitz, 2015, Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupil-linked arousal, Neuron, 85, 628, 10.1016/j.neuron.2014.12.053 · DOI · Paper page
- SG Walling, 2011, Selective tuning of hippocampal oscillations by phasic locus coeruleus activation in awake male rats, Hippocampus, 21, 1250, 10.1002/hipo.20816 · DOI
- MR Nassar, 2012, Rational regulation of learning dynamics by pupil-linked arousal systems, Nat Neurosci
- T Takeuchi, 2016, Locus coeruleus and dopaminergic consolidation of everyday memory Studies of memory for over a century, Nature, 537, 5, 10.1038/nature19325 · DOI · Paper page
- S Hirschberg, 2017, Functional dichotomy in spinal- vs prefrontal-projecting locus coeruleus modules splits descending noradrenergic analgesia from ascending aversion and anxiety in rats, Elife, 6, 10.7554/eLife.29808 · DOI · Paper page
- L Hurley, 2004, A matter of focus: monoaminergic modulation of stimulus coding in mammalian sensory networks, Curr Opin Neurobiol, 14, 488, 10.1016/j.conb.2004.06.007 · DOI · Paper page
- A Uematsu, 2017, Modular organization of the brainstem noradrenaline system coordinates opposing learning states, Nat Neurosci
- M Usher, 1999, The Role of Locus Coeruleus in the Regulation of Cognitive Performance, Science, 283, 5401, 10.1126/science.283.5401.549 · DOI · Paper page
- EC Clayton, 2004, Phasic Activation of Monkey Locus Ceruleus Neurons by Simple Decisions in a Forced-Choice Task, J Neurosci, 24, 9914, 10.1523/JNEUROSCI.2446-04.2004 · DOI · Paper page
- P Dayan, 2006, Phasic norepinephrine: A neural interrupt signal for unexpected events, Netw Comput Neural Syst, 17, 335, 10.1080/09548980601004024 · DOI · Paper page
Cited by
Locus coeruleus: a new look at the blue spot
Poe, G. R., Foote, S., Eschenko, O., Johansen, J. P., Bouret, S., Aston-Jones, G., Harley, C. W., Manahan-Vaughan, D., Weinshenker, D., Valentino, R., et al. (2020) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-0360-9 · Cited by 720
Neuromodulatory Influences on Integration and Segregation in the Brain
Shine, J. M. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.04.002 · Cited by 317
A step-by-step tutorial on active inference and its application to empirical data
Smith, R., Friston, K. J., & Whyte, C. J. (2022) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2021.102632 · Cited by 246
Self-citationGeneralised free energy and active inference
Parr, T., & Friston, K. J. (2019) · Biological Cybernetics · doi:10.1007/s00422-019-00805-w · Cited by 236
Self-citationCorcoran, A. W., Pezzulo, G., & Hohwy, J. (2020) · Biology & Philosophy · doi:10.1007/s10539-020-09746-2 · Cited by 161
The Misestimation of Uncertainty in Affective Disorders
Pulcu, E., & Browning, M. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.07.007 · Cited by 156
“Consistent with the putative role of central NE in representing estimated unexpected uncertainty [1,17,18], pupil dilation was found to be greater when outcomes were presented in the volatile than in the stable block, with the size of this volatility-related pupillary effect also being significantly smaller in anxious…”
Reward maximization through discrete active inference
Da Costa, L., Sajid, N., Parr, T., Friston, K., & Smith, R. (2023) · Neural Computation · doi:10.1162/neco_a_01574 · Cited by 35
Self-citationSuicide prevention and ketamine: insights from computational modeling
Charlton, C. E., Karvelis, P., McIntyre, R. S., & Diaconescu, A. O. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1214018 · Cited by 7
Evidence and implications of abnormal predictive coding in dementia
Kocagoncu, E., Klimovich-Gray, A., Hughes, L. E., & Rowe, J. B. (2021) · Brain · doi:10.1093/brain/awab254 · Cited by 3
“‘surprise’).(115) The locus coeruleus is affected by Alzheimer’s disease, Parkinson’s disease and frontotemporal lobar degeneration,(111) which has led to noradrenergic treatment strategies to reduce impulsivity.”
Background
Metadata
- Authors
- Sales, A. C., Friston, K. J., Jones, M. W., Pickering, A. E., & Moran, R. J.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-01-04
- Volume
- 15
- Issue
- 1
- Pages
- e1006267
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2953321017
- Cited by
- 145
- References
- 33
- FWCI
- 7.17
- Type
- journal article
- Language
- en
- Update notices
- new_version (2019-01-16)
- Open access
- gold
- Funders
- Wellcome Trust (108899/Z/15/Z, 088373/Z/09/A, 088130/Z/09/Z)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.