The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Pupil Diameter Predicts Changes in the Exploration–Exploitation Trade-off: Evidence for the Adaptive Gain Theory
Abstract
The adaptive regulation of the balance between exploitation and exploration is critical for the optimization of behavioral performance. Animal research and computational modeling have suggested that changes in exploitative versus exploratory control state in response to changes in task utility are mediated by the neuromodulatory locus coeruleus–norepinephrine (LC–NE) system. Recent studies have suggested that utility-driven changes in control state correlate with pupil diameter, and that pupil diameter can be used as an indirect marker of LC activity. We measured participants' pupil diameter while they performed a gambling task with a gradually changing payoff structure. Each choice in this task can be classified as exploitative or exploratory using a computational model of reinforcement learning. We examined the relationship between pupil diameter, task utility, and choice strategy (exploitation vs. exploration), and found that (i) exploratory choices were preceded by a larger baseline pupil diameter than exploitative choices; (ii) individual differences in baseline pupil diameter were predictive of an individual's tendency to explore; and (iii) changes in pupil diameter surrounding the transition between exploitative and exploratory choices correlated with changes in task utility. These findings provide novel evidence that pupil diameter correlates closely with control state, and are consistent with a role for the LC–NE system in the regulation of the exploration–exploitation trade-off in humans.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
The role of the medial frontal cortex in cognitive control
Ridderinkhof, K. R., Ullsperger, M., Crone, E. A., & Nieuwenhuis, S. (2004) · Science · doi:10.1126/science.1100301 · Cited by 2,974
Self-citationBerridge, C. W., & Waterhouse, B. D. (2003) · Brain Research Reviews · doi:10.1016/s0165-0173(03)00143-7 · Cited by 2,529 · cited 2×
“The ensuing phasic release of NE in cortical areas temporarily increases the responsivity (or gain) of these areas to their afferent input, selectiv ly potentiating the processing of these taskrelevant stimuli (Berridge & Waterhouse, 2003; Doya, 2002; Servan-Schreiber, Printz, & Cohen, 1990).”
BackgroundCortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Advances in Neural Information Processing Systems 18
Murray, I., Mackay, D., Zoubin Ghahramani, A., & Skilling, J. (2006) · Cited by 1,927
The locus coeruleus and noradrenergic modulation of cognition
Sara, S. J. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2573 · Cited by 1,848
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · cited 2×
“Only recently, researchers ave begun to examine its involvement in more specific cognitive functions, such as attention, memory, perceptual selection and the signaling of unexpected uncertainty (Einhäuser, Stout, Koch, & Carter, 2008; Robbins, 1997; Sara, 2009; Yu and Dayan, 2005).”
BackgroundInfluentialSutton, R. S. (1990) · Machine Learning Proceedings 1990 · doi:10.1016/b978-1-55860-141-3.50030-4 · Cited by 1,314
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121
Conflict monitoring and decision making: Reconciling two perspectives on anterior cingulate function
BOTVINICK, M. M. (2007) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.7.4.356 · Cited by 1,117
Orienting and reorienting: the locus coeruleus mediates cognition through arousal
Sara, S. J., & Bouret, S. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.011 · Cited by 1,056
Gilzenrat, M. S., Nieuwenhuis, S., Jepma, M., & Cohen, J. D. (2010) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.10.2.252 · Cited by 798
Self-citationRegression analyses of repeated measures data in cognitive research.
Lorch, R. F., & Myers, J. L. (1990) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037//0278-7393.16.1.149 · Cited by 791
The Brain Nucleus Locus Coeruleus: Restricted Afferent Control of a Broad Efferent Network
Aston-Jones, G., Ennis, M., Pieribone, V. A., Nickell, W. T., & Shipley, M. T. (1986) · Science · doi:10.1126/science.3775363 · Cited by 778
A Network Model of Catecholamine Effects: Gain, Signal-to-Noise Ratio, and Behavior
Servan-Schreiber, D., Printz, H., & Cohen, J. D. (1990) · Science · doi:10.1126/science.2392679 · Cited by 767
Rational regulation of learning dynamics by pupil-linked arousal systems
Nassar, M. R., Rumsey, K. M., Wilson, R. C., Parikh, K., Heasly, B., & Gold, J. I. (2012) · Nature Neuroscience · doi:10.1038/nn.3130 · Cited by 722
The role of locus coeruleus in the regulation of cognitive performance
Usher, M., Cohen, J. D., Servan-Schreiber, D., Rajkowski, J., & Aston-Jones, G. (1999) · Science · doi:10.1126/science.283.5401.549 · Cited by 715
Metalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704
Boorman, E. D., Behrens, T. E. J., Woolrich, M. W., & Rushworth, M. F. S. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.014 · Cited by 665 · cited 1×
“In addition to other neuromodulator systems, recent studies have implicated fronto-polar cortex in the control of exploratory behaviors (Bourdaud, Chavarriaga, Galan, & Millan, 2008 ; Daw et al, 2006 ), although the specific computations performed by fronto-polar cortex in this context are a topic of ongoing debate (B…”
BackgroundThe Pupil: Anatomy, Physiology, and Clinical Applications
Loewenfeld, I. E., & Lowenstein, O. E. (1999) · Cited by 627
Neurochemical Modulation of Response Inhibition and Probabilistic Learning in Humans
Chamberlain, S. R., Müller, U., Blackwell, A. D., Clark, L., Robbins, T. W., & Sahakian, B. J. (2006) · Science · doi:10.1126/science.1121218 · Cited by 573 · cited 2×
“…characterize the tonic LC mode) have been found to improve attentional-set shifting and reversal learning in rats and monkeys (Devauges & Sara, 1990; Lapiz & Morilak, 2006; Lapiz, Bondi, & Morilak, 2007; Seu, Lang, Rivera, & Jentsch, 2008; Steere & Arnsten, 1997; but see Chamberlain et al., 2006).”
BackgroundPupillometry and P3 index the locus coeruleus-noradrenergic arousal function in humans
Murphy, P. R., Robertson, I. H., Balsters, J. H., & O'connell, R. G. (2011) · Psychophysiology · doi:10.1111/j.1469-8986.2011.01226.x · Cited by 519
Frank, M. J., Doll, B. B., Oas-Terpstra, J., & Moreno, F. (2009) · Nature Neuroscience · doi:10.1038/nn.2342 · Cited by 471
Nieuwenhuis, S., De Geus, E. J., & Aston‐Jones, G. (2011) · Psychophysiology · doi:10.1111/j.1469-8986.2010.01057.x · Cited by 449
Self-citationAnterior cingulate activity during error and autonomic response
Critchley, H. D., Tang, J., Glaser, D., Butterworth, B., & Dolan, R. J. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.05.047 · Cited by 445
Arousal systems and attentional processes
Robbins, T. W. (1997) · Biological Psychology · doi:10.1016/s0301-0511(96)05222-2 · Cited by 430
Locus coeruleus activity in monkey: Phasic and tonic changes are associated with altered vigilance
Rajkowski, J., Kubiak, P., & Aston-Jones, G. (1994) · Brain Research Bulletin · doi:10.1016/0361-9230(94)90175-9 · Cited by 371
Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry
Einhäuser, W., Stout, J., Koch, C., & Carter, O. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0707727105 · Cited by 351
The Pupil. Anatomy, Physiology and Clinical Applications.
DIAMOND, J. P. (2001) · British Journal of Ophthalmology · doi:10.1136/bjo.85.1.121e · Cited by 332
Lapiz, M. D. S., & Morilak, D. A. (2006) · Neuroscience · doi:10.1016/j.neuroscience.2005.09.031 · Cited by 325
Pupil dilation deconvolution reveals the dynamics of attention at high temporal resolution
Wierda, S. M., van Rijn, H., Taatgen, N. A., & Martens, S. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1201858109 · Cited by 321
A noradrenergic theory of cognitive reserve: implications for Alzheimer's disease
Robertson, I. H. (2013) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2012.05.019 · Cited by 241
İnsanlarda Yenilik N2 Yanıtı Hedef Uyaranların Zamansal Sınıflamasını Yansıtır
MARAŞLIGİL, B., & ERGENOĞLU, T. (2011) · Journal of Experimental and Clinical Medicine · doi:10.5835/jecm.omu.28.02.002 · Cited by 235
Aston-Jones, G., Rajkowski, J., & Kubiak, P. (1997) · Neuroscience · doi:10.1016/s0306-4522(97)00060-2 · Cited by 229
Use of concurrent pupil dilation assessment to inform interpretation and analysis of fMRI data
Siegle, G. J., Steinhauer, S. R., Stenger, V. A., Konecky, R., & Carter, C. S. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00298-2 · Cited by 220
Kuchinsky, S. E., Ahlstrom, J. B., Vaden, K. I., Cute, S. L., Humes, L. E., Dubno, J. R., & Eckert, M. A. (2013) · Psychophysiology · doi:10.1111/j.1469-8986.2012.01477.x · Cited by 203
Activation of the noradrenergic system facilitates an attentional shift in the rat
Devauges, V., & Sara, S. J. (1990) · Behavioural Brain Research · doi:10.1016/0166-4328(90)90118-x · Cited by 200
Dreisbach, G., Müller, J., Goschke, T., Strobel, A., Schulze, K., Lesch, K.-P., & Brocke, B. (2005) · Behavioral Neuroscience · doi:10.1037/0735-7044.119.2.483 · Cited by 194
Satterthwaite, T. D., Green, L., Myerson, J., Parker, J. B., Ramaratnam, M., & Buckner, R. L. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.04.066 · Cited by 171
(1993) · Medical Entomology and Zoology · Cited by 168
Abercrombie, E., & Jacobs, B. (1987) · The Journal of Neuroscience · doi:10.1523/jneurosci.07-09-02844.1987 · Cited by 166
Pupillometry: The Psychology of the Pupillary Response
Hess, E. H., & Janisse, M. P. (1978) · The American Journal of Psychology · doi:10.2307/1421703 · Cited by 136
Inhibition of the norepinephrine transporter improves behavioral flexibility in rats and monkeys
Seu, E., Lang, A., Rivera, R. J., & Jentsch, J. D. (2008) · Psychopharmacology · doi:10.1007/s00213-008-1250-4 · Cited by 131
Reiner, P. B. (1986) · Brain Research · doi:10.1016/0006-8993(86)90288-x · Cited by 114
Locus coeruleus neurons and sympathetic nerves: Activation by cutaneous sensory afferents
Elam, M., Svensson, T. H., & Thorén, P. (1986) · Brain Research · doi:10.1016/0006-8993(86)91302-8 · Cited by 102
Lapiz, M. D. S., Bondi, C. O., & Morilak, D. A. (2007) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301235 · Cited by 95
Steere, J. C., & Arnsten, A. F. T. (1997) · Behavioral Neuroscience · doi:10.1037/0735-7044.111.5.883 · Cited by 94 · cited 1×
“…characterize the tonic LC mode) have been found to improve attentional-set shifting and reversal learning in rats and monkeys (Devauges & Sara, 1990; Lapiz & Morilak, 2006; Lapiz, Bondi, & Morilak, 2007; Seu, Lang, Rivera, & Jentsch, 2008; Steere & Arnsten, 1997; but see Chamberlain et al., 2006).”
BackgroundSilvetti, M., Seurinck, R., & Verguts, T. (2012) · Cortex · doi:10.1016/j.cortex.2012.05.008 · Cited by 92
Difference in serotonergic and noradrenergic regulation of human social behaviours
Tse, W. S., & Bond, A. J. (2001) · Psychopharmacology · doi:10.1007/s00213-001-0926-9 · Cited by 79
Kashino, M., & Kondo, H. M. (2012) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2011.0370 · Cited by 66
Blinking predicts enhanced cognitive control
van Bochove, M. E., der Haegen, L. V., Notebaert, W., & Verguts, T. (2012) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-012-0138-2 · Cited by 65
An exploration-exploitation model based on norepinepherine and dopamine activity
McClure, S. M., Gilzenrat, M. S., & Cohen, J. D. (2005) · neural information processing systems · Cited by 61
The timing of exploratory decision-making revealed by single-trial topographic EEGanalyses
Tzovara, A., Murray, M. M., Bourdaud, N., Chavarriaga, R., Millán, J. D. R., & Marzia De Lucia, E. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.136 · Cited by 44
Characterizing the EEG Correlates of Exploratory Behavior
Bourdaud, N., Chavarriaga, R., Galán, F., & Millán, J. D. R. (2008) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2008.926712 · Cited by 34
Change is in the eye of the beholder
Yu, A. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3150 · Cited by 24
Cited by
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321
Joshi, S., Li, Y., Kalwani, R. M., & Gold, J. I. (2016) · Neuron · doi:10.1016/j.neuron.2015.11.028 · Cited by 1,387
An opportunity cost model of subjective effort and task performance
Kurzban, R., Duckworth, A., Kable, J. W., & Myers, J. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12003196 · Cited by 1,381
Toward a rational and mechanistic account of mental effort
Shenhav, A., Musslick, S., Lieder, F., Kool, W., Griffiths, T. L., Cohen, J. D., & Botvinick, M. M. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031526 · Cited by 1,077
Orienting and reorienting: the locus coeruleus mediates cognition through arousal
Sara, S. J., & Bouret, S. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.011 · Cited by 1,056
Metadata
- Authors
- Jepma, M., & Nieuwenhuis, S.
- Venue
- Journal of Cognitive Neuroscience
- Publisher
- MIT Press - Journals
- Published
- 2011-07-01
- Volume
- 23
- Issue
- 7
- Pages
- 1587-1596
- ISSN
- 0898-929X (ISSN-L)
- 1530-8898
- OpenAlex
- W2129882844
- Cited by
- 439
- References
- 56
- FWCI
- 7.24
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.