Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration
Open accessAbstract
Many large and small decisions we make in our daily lives-which ice cream to choose, what research projects to pursue, which partner to marry-require an exploration of alternatives before committing to and exploiting the benefits of a particular choice. Furthermore, many decisions require re-evaluation, and further exploration of alternatives, in the face of changing needs or circumstances. That is, often our decisions depend on a higher level choice: whether to exploit well known but possibly suboptimal alternatives or to explore risky but potentially more profitable ones. How adaptive agents choose between exploitation and exploration remains an important and open question that has received relatively limited attention in the behavioural and brain sciences. The choice could depend on a number of factors, including the familiarity of the environment, how quickly the environment is likely to change and the relative value of exploiting known sources of reward versus the cost of reducing uncertainty through exploration. There is no known generally optimal solution to the exploration versus exploitation problem, and a solution to the general case may indeed not be possible. However, there have been formal analyses of the optimal policy under constrained circumstances. There have also been specific suggestions of how humans and animals may respond to this problem under particular experimental conditions as well as proposals about the brain mechanisms involved. Here, we provide a brief review of this work, discuss how exploration and exploitation may be mediated in the brain and highlight some promising future directions for research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The learning rate β depends on global task properties such as volatility or uncertainty and is suggested to be modulated by noradrenergic inputs.
“This rate depends on global task properties such as the volatility or uncertainty of a given task, and has been suggested to be modulated by noradrenergic inputs (Cohen et al., 2007; Nassar et al., 2012).”
From Attention, learning, and the value of information (2012) · cites this paper as Cohen et al., 2007
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,037 · see citing works
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803 · see citing works
Self-citationA neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · see citing works
Conflict monitoring and cognitive control
Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001) · Psychological Review · doi:10.1037/0033-295x.108.3.624 · Cited by 7,351 · see citing works
Self-citationAxten, N., Newell, A., & Simon, H. A. (1973) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2063712 · Cited by 6,883 · see citing works
Newell, A. (1972) · Cited by 6,222 · see citing works
Taking time seriously: A theory of socioemotional selectivity.
Carstensen, L. L., Isaacowitz, D. M., & Charles, S. T. (1999) · American Psychologist · doi:10.1037//0003-066x.54.3.165 · Cited by 5,377 · cited 1× · see citing works
“Humans show a greater tendency to explore when there is more time left in a task, presumably because this allows them sufficient time later to enjoy the fruits of those explorations (Carstensen et al. 1999).”
BackgroundAston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472 · see citing works
Self-citationHolroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027 · see citing works
Attention and the detection of signals.
Posner, M. I., Snyder, C. R., & Davidson, B. J. (1980) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.109.2.160 · Cited by 3,778 · see citing works
Anterior Cingulate Cortex, Error Detection, and the Online Monitoring of Performance
Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M., Noll, D., & Cohen, J. D. (1998) · Science · doi:10.1126/science.280.5364.747 · Cited by 3,273 · see citing works
Self-citationCosts of a predictible switch between simple cognitive tasks.
Rogers, R. D., & Monsell, S. (1995) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.124.2.207 · Cited by 3,170 · see citing works
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089 · see citing works
Self-citationAnterior Cingulate Conflict Monitoring and Adjustments in Control
Kerns, J. G., Cohen, J. D., MacDonald, A. W., Cho, R. Y., Stenger, V. A., & Carter, C. S. (2004) · Science · doi:10.1126/science.1089910 · Cited by 2,891 · see citing works
Self-citationSpecious reward: A behavioral theory of impulsiveness and impulse control.
Ainslie, G. (1975) · Psychological Bulletin · doi:10.1037/h0076860 · Cited by 2,708 · see citing works
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · see citing works
Functional imaging of brain responses to pain. A review and meta-analysis (2000)
Peyron, R., Laurent, B., & García-Larrea, L. (2000) · Neurophysiologie Clinique/Clinical Neurophysiology · doi:10.1016/s0987-7053(00)00227-6 · Cited by 2,171 · see citing works
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Abstract reward and punishment representations in the human orbitofrontal cortex
O'Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001) · Nature Neuroscience · doi:10.1038/82959 · Cited by 2,003 · see citing works
The neural basis of error detection: Conflict monitoring and the error-related negativity
Yeung, N., Botvinick, M. M., & Cohen, J. D. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.931 · Cited by 1,924 · see citing works
Self-citationUncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · see citing works
Self-citationNeurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598 · see citing works
Optimizing the use of information: Strategic control of activation of responses.
Gratton, G., COLES, M. G. H., & Donchin, E. (1992) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.121.4.480 · Cited by 1,574 · see citing works
Miltner, W. H. R., Braun, C. H., & Coles, M. G. H. (1997) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1997.9.6.788 · Cited by 1,530 · see citing works
Optimizing the use of information: Strategic control of activation of responses.
Gratton, G., COLES, M. G. H., & Donchin, E. (1992) · Journal of Experimental Psychology General · doi:10.1037//0096-3445.121.4.480 · Cited by 1,514 · see citing works
Bandit processes and dynamic allocation indices
Gittins, J. C. (1979) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1979.tb01068.x · Cited by 1,487 · see citing works
Shifting Intentional Set: Exploring the Dynamic Control of Tasks
(1994) · The MIT Press eBooks · doi:10.7551/mitpress/1478.003.0025 · Cited by 1,317 · see citing works
Matching Behavior and the Representation of Value in the Parietal Cortex
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004) · Science · doi:10.1126/science.1094765 · Cited by 1,030 · see citing works
Errors and error correction in choice-response tasks.
Rabbitt, P. (1966) · Journal of Experimental Psychology · doi:10.1037/h0022853 · Cited by 1,024 · see citing works
Computational roles for dopamine in behavioural control
Montague, P. R., Hyman, S. E., & Cohen, J. D. (2004) · Nature · doi:10.1038/nature03015 · Cited by 966 · see citing works
Self-citationParsing executive processes: Strategic vs. evaluative functions of the anterior cingulate cortex
Carter, C. S., Macdonald, A. M., Botvinick, M., Ross, L. L., Stenger, V. A., Noll, D. C., & Cohen, J. D. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.97.4.1944 · Cited by 954 · see citing works
Self-citationCollective Motion, Sensor Networks, and Ocean Sampling
Leonard, N. E., Paley, D. A., Lekien, F., Sepulchre, R. J., Fratantoni, D. M., & Davis, R. E. (2007) · Proceedings of the IEEE · doi:10.1109/jproc.2006.887295 · Cited by 947 · see citing works
Knutson, B. D., Fong, G., Bennett, S. M., Adams, C. M., & Hommer, D. W. (2003) · NeuroImage · doi:10.1016/s1053-8119(02)00057-5 · Cited by 850 · see citing works
(1994) · The MIT Press eBooks · doi:10.7551/mitpress/1478.001.0001 · Cited by 749 · see citing works
The role of locus coeruleus in the regulation of cognitive performance
Usher, M., Cohen, J. D., Servan-Schreiber, D., Rajkowski, J., & Aston-Jones, G. (1999) · Science · doi:10.1126/science.283.5401.549 · Cited by 715 · see citing works
Self-citationHandbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches
White, D. A., & Sofge, D. (1992) · Medical Entomology and Zoology · Cited by 705 · see citing works
Berry, D. A., & Fristedt, B. E. (1985) · Springer eBooks · doi:10.1007/978-94-015-3711-7 · Cited by 645 · see citing works
Locus coeruleus neurons in monkey are selectively activated by attended cues in a vigilance task
Aston-Jones, G., Rajkowski, J., Kubiak, P., & Alexinsky, T. (1994) · The Journal of Neuroscience · doi:10.1523/jneurosci.14-07-04467.1994 · Cited by 637 · see citing works
The Neural Basis of Error Detection: Conflict Monitoring and the Error-Related Negativity.
Yeung, N., Botvinick, M., & Cohen, J. D. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.939 · Cited by 616 · see citing works
Self-citationKaelbling, L. P. (1993) · doi:10.7551/mitpress/4168.001.0001 · Cited by 599 · see citing works
Test of optimal sampling by foraging great tits
Krebs, J. R., Kacelnik, A., & Taylor, P. (1978) · Nature · doi:10.1038/275027a0 · Cited by 599 · see citing works
Bandit Problems, Sequential Allocation of Experiments.
Jones, P. W., Berry, D. A., & Fristedt, B. (1987) · Journal of the Operational Research Society · doi:10.2307/2582852 · Cited by 583 · see citing works
A computational substrate for incentive salience
McClure, S. M., Daw, N. D., & Montague, P. R. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(03)00177-2 · Cited by 416 · see citing works
Self-citationIMAGING VALUATION MODELS IN HUMAN CHOICE
Montague, P. R., King‐Casas, B., & Cohen, J. D. (2006) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112903 · Cited by 371 · see citing works
Self-citationChoice reaction performance following an error
Laming, D. (1979) · Acta Psychologica · doi:10.1016/0001-6918(79)90026-x · Cited by 263 · see citing works
The role of exploration in learning control
Thrun, S. (1992) · Cited by 227 · see citing works
Aston‐Jones, G., Rajkowski, J., & Kubiak, P. (1997) · Neuroscience · doi:10.1016/s0306-4522(97)00060-2 · Cited by 227 · see citing works
Autocorrelation of choice-reaction times
Laming, D. (1979) · Acta Psychologica · doi:10.1016/0001-6918(79)90032-5 · Cited by 166 · see citing works
A tunable algorithm for collective decision-making
Pratt, S. C., & Sumpter, D. J. T. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0604801103 · Cited by 148 · see citing works
Switching Costs and the Gittins Index
Banks, J. S., & Sundaram, R. K. (1994) · Econometrica · doi:10.2307/2951664 · Cited by 130 · see citing works
SIMPLE NEURAL NETWORKS THAT OPTIMIZE DECISIONS
Brown, E., Gao, J., Holmes, P. J., Bogacz, R., Gilzenrat, M. S., & Cohen, J. D. (2005) · International Journal of Bifurcation and Chaos · doi:10.1142/s0218127405012478 · Cited by 101 · see citing works
Self-citationAn exploration-exploitation model based on norepinepherine and dopamine activity
McClure, S. M., Gilzenrat, M. S., & Cohen, J. D. (2005) · neural information processing systems · Cited by 61 · see citing works
Self-citationA Neural Substrate of Prediction and Reward
Schultz, W., Dayan, P., & Montague, P. R. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/3077.003.0038 · Cited by 53 · see citing works
New approaches to investigating the function of mycelial networks
Watkinson, S., Boddy, L., Burton, K. S., et al. (2005) · Mycologist · doi:10.1017/s0269915x05001023 · Cited by 36 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Aston-Jones G, Rajkowski J, Lu W, Zhu Y, Cohen J.D, Morecraft R.J. Prominent projections from the orbital prefrontal cortex to the locus coeruleus in monkeys. Soc. Neurosci. Abstr. 2002;28:86–89. [Google Scholar]
- Gittins J.C, Jones D.M. A dynamic allocation index for the sequential design of experiments. In: Gans J, editor. Progress in statisics. North-Holland; Amsterdam, The Netherlands: 1974. pp. 241–266. [Google Scholar]
- Rajkowski J, Lu W, Zhu Y, Cohen J.D, Aston-Jones G. Prominent projections from the anterior cingulate cortex to the locus coeruleus in Rhesus monkey. Soc. Neurosci. Abstr. 2000;26:838.15. [Google Scholar]
Cited by
Sorted by citations
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · see citing works
An opportunity cost model of subjective effort and task performance
Kurzban, R., Duckworth, A., Kable, J. W., & Myers, J. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12003196 · Cited by 1,381 · see citing works
Human volition: towards a neuroscience of will
Haggard, P. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2497 · Cited by 1,134 · see citing works
Active inference and epistemic value
Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., & Pezzulo, G. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1020053 · Cited by 731 · see citing works
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580 · see citing works
Pupillometry and P3 index the locus coeruleus-noradrenergic arousal function in humans
Murphy, P. R., Robertson, I. H., Balsters, J. H., & O'connell, R. G. (2011) · Psychophysiology · doi:10.1111/j.1469-8986.2011.01226.x · Cited by 519 · see citing works
The entropic brain - revisited
Carhart-Harris, R. L. (2018) · Neuropharmacology · doi:10.1016/j.neuropharm.2018.03.010 · Cited by 522 · see citing works
Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making
Collins, A., & Koechlin, E. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001293 · Cited by 558 · see citing works
“The results therefore reveal two fundamentally distinct human exploration processes: first, uncontrolled exploration stochastically selecting actions within actor task sets according to a softmax policy for learning behavioral strategies that maximize action utility [3,28,41], and second, controlled exploration occurr…”
BackgroundJones, M., & Love, B. C. (2011) · Behavioral and Brain Sciences · doi:10.1017/s0140525x10003134 · Cited by 533 · see citing works
Thinking, fast and slow? Some field experiments to reduce crime and dropout in Chicago
Heller, S. B., Shah, A. K., Guryan, J., Ludwig, J., Mullainathan, S., & Pollack, H. A. (2017) · The Quarterly Journal of Economics · doi:10.1093/qje/qjw033 · Cited by 493 · see citing works
“…versus exploring potentially more rewarding strategies.11 To make this more concrete, imagine a relative promises to pick up a teenager from school, 11 This is known in reinforcement learning as the “exploration versus exploitation” problem (see e.g., Kaelbling et al., 1996, Cohen et al., 2007).”
BackgroundTowards a neuroscience of active sampling and curiosity
Gottlieb, J., & Oudeyer, P.-Y. (2018) · Nature Reviews Neuroscience · doi:10.1038/s41583-018-0078-0 · Cited by 494 · see citing works
The role of privacy fatigue in online privacy behavior
Choi, H., Park, J., & Jung, Y. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2017.12.001 · Cited by 427 · see citing works
The neuromodulator of exploration: A unifying theory of the role of dopamine in personality
DeYoung, C. G. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00762 · Cited by 387 · see citing works
A computational framework for the study of confidence in humans and animals
Kepecs, A., & Mainen, Z. F. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0037 · Cited by 305 · see citing works
The mesoaccumbens dopamine in coping with stress
Cabib, S., & Puglisi-Allegra, S. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2011.04.012 · Cited by 329 · see citing works
The anatomy of choice: active inference and agency
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00598 · Cited by 340 · see citing works
Hierarchical active inference: A theory of motivated control
Pezzulo, G., Rigoli, F., & Friston, K. J. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.01.009 · Cited by 334 · see citing works
Grounding cognitive control in associative learning
Abrahamse, E., Braem, S., Notebaert, W., & Verguts, T. (2016) · Psychological Bulletin · doi:10.1037/bul0000047 · Cited by 276 · see citing works
The anatomy of choice: dopamine and decision-making.
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0481 · Cited by 289 · see citing works
Internal state dynamics shape brainwide activity and foraging behaviour
Marques, J. C., Li, M., Schaak, D., Robson, D. N., & Li, J. M. (2020) · Nature · doi:10.1038/s41586-019-1858-z · Cited by 243 · see citing works
Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Payzan-LeNestour, E., & Bossaerts, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001048 · Cited by 267 · see citing works
Dopamine modulates novelty seeking behavior during decision making.
Costa, V. D., Tran, V. L., Turchi, J., & Averbeck, B. B. (2014) · Behavioral Neuroscience · doi:10.1037/a0037128 · Cited by 263 · see citing works
Deconstructing the human algorithms for exploration
Gershman, S. J. (2018) · Cognition · doi:10.1016/j.cognition.2017.12.014 · Cited by 267 · see citing works
Basal Ganglia Circuits for Reward Value–Guided Behavior
Hikosaka, O., Kim, H. F., Yasuda, M., & Yamamoto, S. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071013-013924 · Cited by 259 · see citing works
Attention, learning, and the value of information
Gottlieb, J. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.034 · Cited by 241 · see citing works
“This rate may depend on global task properties such as the volatility or uncertainty of a given task and could be conveyed through neuromodulation (Cohen et al., 2007; Nas-sar et al., 2012).”
BackgroundThe dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219 · see citing works
Minds and brains of media multitaskers: Current findings and future directions
Uncapher, M. R., & Wagner, A. D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611612115 · Cited by 217 · see citing works
Abstract context representations in primate amygdala and prefrontal cortex
Saez, A., Rigotti, M., Ostojic, S., Fusi, S., & Salzman, C. D. (2015) · Neuron · doi:10.1016/j.neuron.2015.07.024 · Cited by 183 · see citing works
The sense of should: A biologically-based framework for modeling social pressure
Theriault, J. E., Young, L., & Barrett, L. F. (2021) · Physics of Life Reviews · doi:10.1016/j.plrev.2020.01.004 · Cited by 201 · see citing works
Cognitive Control: Core Constructs and Current Considerations
Cohen, J. D. (2017) · The Wiley Handbook of Cognitive Control · doi:10.1002/9781118920497.ch1 · Cited by 179 · see citing works
Self-citationHauser, T. U., Iannaccone, R., Ball, J., Mathys, C., Brandeis, D., Walitza, S., & Brem, S. (2014) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.1093 · Cited by 168 · see citing works
Pezzulo, G., Rigoli, F., & Chersi, F. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00092 · Cited by 163 · see citing works
The human amygdala and orbital prefrontal cortex in behavioural regulation
Dolan, R. J. (2007) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2088 · Cited by 150 · see citing works
Adaptive effort investment in cognitive and physical tasks: A neurocomputational model
Verguts, T., Vassena, E., & Silvetti, M. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2015.00057 · Cited by 118 · see citing works
Vive les differences! Individual variation in neural mechanisms of executive control
Braver, T. S., Cole, M. W., & Yarkoni, T. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.03.002 · Cited by 130 · see citing works
Role of nicotinic acetylcholine receptors in regulating dopamine neuron activity
Faure, P., Tolu, S., Valverde, S., & Naudé, J. (2014) · Neuroscience · doi:10.1016/j.neuroscience.2014.05.040 · Cited by 125 · see citing works
The anterior cingulate cortex directs exploration of alternative strategies
Tervo, D. G. R., Kuleshova, E., Manakov, M., Proskurin, M., Karlsson, M., Lustig, A., Behnam, R., & Karpova, A. Y. (2021) · Neuron · doi:10.1016/j.neuron.2021.03.028 · Cited by 97 · see citing works
Vinckier, F., Gaillard, R., Palminteri, S., Rigoux, L., Salvador, A., Fornito, A., Adapa, R., Krebs, M. O., Pessiglione, M., & Fletcher, P. C. (2016) · Molecular Psychiatry · doi:10.1038/mp.2015.73 · Cited by 87 · see citing works
Cognitive inflexibility in OCD and related disorders
Chamberlain, S. R., Solly, J. E., Hook, R. W., Vaghi, M. M., & Robbins, T. W. (2021) · Current Topics in Behavioral Neurosciences · doi:10.1007/7854_2020_198 · Cited by 82 · see citing works
Novelty–variety as a candidate basic psychological need: New evidence across three studies
Bagheri, L., & Milyavskaya, M. (2020) · Motivation and Emotion · doi:10.1007/s11031-019-09807-4 · Cited by 74 · see citing works
Medaglia, J. D., Satterthwaite, T. D., Kelkar, A., Ciric, R., Moore, T. M., Ruparel, K., Gur, R. C., Gur, R. E., & Bassett, D. S. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.10.048 · Cited by 64 · see citing works
Balancing new against old information: the role of puzzlement surprise in learning
Faraji, M., Preuschoff, K., & Gerstner, W. (2018) · Neural Computation · doi:10.1162/neco_a_01025 · Cited by 68 · see citing works
Hervig, M. E., Fiddian, L., Piilgaard, L., Božič, T., Blanco-Pozo, M., Knudsen, C., Olesen, S. F., Alsiö, J., & Robbins, T. W. (2020) · Cerebral Cortex · doi:10.1093/cercor/bhz144 · Cited by 65 · see citing works
Jepma (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00170 · Cited by 45 · see citing works
Active learning: learning a motor skill without a coach
Huang, V. S., Shadmehr, R., & Diedrichsen, J. (2008) · Journal of Neurophysiology · doi:10.1152/jn.01095.2007 · Cited by 41 · see citing works
Computational psychopharmacology: a translational and pragmatic approach
Robbins, T. W., & Cardinal, R. N. (2019) · Psychopharmacology · doi:10.1007/s00213-019-05302-3 · Cited by 25 · see citing works
Brain network dynamics during spontaneous strategy shifts and incremental task optimization
Allegra, M., Seyed-Allaei, S., Schuck, N. W., Amati, D., Laio, A., & Reverberi, C. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116854 · Cited by 27 · see citing works
Aubert‐Teillaud, B., Bran, A., & Vaidis, D. C. (2023) · European Journal of Social Psychology · doi:10.1002/ejsp.2981 · Cited by 19 · see citing works
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6 · see citing works
Infraslow histaminergic dynamics govern priming states to gate moment-to-moment memory accessibility
Morishita, Y., Takamura, Y., Nishimura, K., Yokoi, Y., Ishihama, Y., Idutsu, R., Ono, M., Matsumoto, R., Hitora-Imamura, N., Minami, M., et al. (2026) · Neuron · doi:10.1016/j.neuron.2026.05.019
- Authors
- Cohen, J. D., McClure, S. M., & Yu, A. J.
- Affiliations
- University of Pittsburgh (US)
- Princeton University (US)
- Corresponding author
- Cohen, Jonathan D
- Venue
- Philosophical Transactions of the Royal Society B Biological Sciences
- Publisher
- The Royal Society
- Published
- 2007-03-29
- Volume
- 362
- Issue
- 1481
- Pages
- 933-942
- ISSN
- 0962-8436 (ISSN-L)
- 1471-2970
- PMC
- PMC2430007
- OpenAlex
- W2134243390
- Cited by
- 1,121
- References
- 57
- Citations by year
- 2026: 79 · 2025: 84 · 2024: 84 · 2023: 72 · 2022: 81 · 2021: 84 · 2020: 77 · 2019: 64 · 2018: 55 · 2017: 68 · 2016: 56 · 2015: 47 · 2014: 70 · 2013: 49 · 2012: 42
- FWCI
- 1.57
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Philosophical Transactions of the Royal Society B Biological Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- TUbilio (Technical University of Darmstadt) (submittedVersion)
- Funders
- National Institute of Mental Health (MH-062196, MH65214, F32MH072141, P50MH062196, T32MH065214)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- exploration-exploitation balance, decision-making, risk-taking, uncertainty, reward, neural mechanisms, adaptive behavior, optimal control, environmental change, familiarity
- MeSH terms
- Brain, Decision Making, Exploratory Behavior, Humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
