Humans use directed and random exploration to solve the explore–exploit dilemma.
Open accessThe DOI matches its Crossref record; 33 of the 34 references it cites were found in a registry or an index.
Abstract
All adaptive organisms face the fundamental tradeoff between pursuing a known reward (exploitation) and sampling lesser-known options in search of something better (exploration). Theory suggests at least two strategies for solving this dilemma: a directed strategy in which choices are explicitly biased toward information seeking, and a random strategy in which decision noise leads to exploration by chance. In this work we investigated the extent to which humans use these two strategies. In our "Horizon task," participants made explore-exploit decisions in two contexts that differed in the number of choices that they would make in the future (the time horizon). Participants were allowed to make either a single choice in each game (horizon 1), or 6 sequential choices (horizon 6), giving them more opportunity to explore. By modeling the behavior in these two conditions, we were able to measure exploration-related changes in decision making and quantify the contributions of the two strategies to behavior. We found that participants were more information seeking and had higher decision noise with the longer horizon, suggesting that humans use both strategies to solve the exploration-exploitation dilemma. We thus conclude that both information seeking and choice variability can be controlled and put to use in the service of exploration.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“People show both of these effects (27).”
From Using cognitive psychology to understand GPT-3 (2023) · cites this paper as [27]
The cited source says what the claim attributes to it.
In the citing paper
“While humans show such an effect (27), we did not find it in GPT-3 (β = −0.15 ± 0.27, z = −0.56, P = .58), which demonstrates that the model did not employ directed exploration.”
From Using cognitive psychology to understand GPT-3 (2023) · cites this paper as [27]
The cited source says what the claim attributes to it.
In the citing paper
“Previous research suggests that people solve this exploration-exploitation trade-off by applying a combination of two distinct strategies: directed and random exploration (27).”
From Using cognitive psychology to understand GPT-3 (2023) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
“Risk, Ambiguity, and the Savage Axioms.”
Ellsberg, D. (1961) · The Quarterly Journal of Economics · doi:10.2307/1884324 · Cited by 7,891
Finite-time Analysis of the Multiarmed Bandit Problem
Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002) · Machine Learning · doi:10.1023/a:1013689704352 · Cited by 6,033
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
Self-citationFaisal, A. A., Selen, L. P. J., & Wolpert, D. M. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2258 · Cited by 2,883
THOMPSON, W. R. (1933) · Biometrika · doi:10.1093/biomet/25.3-4.285 · Cited by 2,712
Pfanzagl, J., & Luce, R. D. (1962) · Econometrica · doi:10.2307/1911299 · Cited by 2,469
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Recent developments in modeling preferences: Uncertainty and ambiguity
Camerer, C., & Weber, M. (1992) · Journal of Risk and Uncertainty · doi:10.1007/bf00122575 · Cited by 1,795
Research works printed but not linked: 2
- Bridle JS. Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimates of parameters. In: Touretzky DS, editor. Advances in neural information processing systems. Vol. 2. Cambridge, MA: MIT Press; 1990. pp. 211–217. [Google Scholar]
- Watkins CJCH. Unpublished doctoral dissertation. Cambridge University; Cambridge, England: 1989. Learning from delayed rewards. [Google Scholar]
Cited by
Items 101–200 of 627
Aberg, K. C., Toren, I., & Paz, R. (2021) · Molecular Psychiatry · doi:10.1038/s41380-021-01363-z · Cited by 53
Dopaminergic genes are associated with both directed and random exploration
Gershman, S. J., & Tzovaras, B. G. (2018) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2018.10.009 · Cited by 51
Distinct Roles of Dopamine and Noradrenaline in Incidental Memory
Hauser, T. U., Eldar, E., Purg, N., Moutoussis, M., & Dolan, R. J. (2019) · Journal of Neuroscience · doi:10.1523/jneurosci.0401-19.2019 · Cited by 48
An empirical evaluation of active inference in multi-armed bandits
Marković, D., Stojić, H., Schwöbel, S., & Kiebel, S. J. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.08.018 · Cited by 45
Geana, A., Barch, D. M., Gold, J. M., Carter, C. S., MacDonald, A. W., Ragland, J. D., Silverstein, S. M., & Frank, M. J. (2021) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2021.03.017 · Cited by 48
Self-citationLigneul, R. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006989 · Cited by 46
Human skill learning: expansion, exploration, selection, and refinement
Lövdén, M., Garzón, B., & Lindenberger, U. (2020) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2020.11.002 · Cited by 50
Strategically managing learning during perceptual decision making
Masis, J. A., Chapman, T., Rhee, J. Y., Cox, D., & Saxe, A. (2023) · eLife · doi:10.7554/elife.64978 · Cited by 47
Developmental Changes in Learning: Computational Mechanisms and Social Influences
Bolenz, F., Reiter, A. M. F., & Eppinger, B. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.02048 · Cited by 47
Jach, H. K., Cools, R., Frisvold, A., Grubb, M. A., Hartley, C. A., Hartmann, J., Hunter, L., Jia, R., de Lange, F. P., Larisch, R., et al. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2415236121 · Cited by 45
Self-citationWyatt, L. E., Hewan, P., Hogeveen, J., Spreng, R. N., & Turner, G. R. (2023) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2023.108740 · Cited by 55
Unifying Principles of Generalization: Past, Present, and Future
Wu, C. M., Meder, B., & Schulz, E. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-021524-110810 · Cited by 47
Temporal discounting correlates with directed exploration but not with random exploration
Sadeghiyeh, H., Wang, S., Alberhasky, M. R., Kyllo, H. M., Shenhav, A., & Wilson, R. C. (2020) · Scientific Reports · doi:10.1038/s41598-020-60576-4 · Cited by 46
Self-citationMeta-learned models of cognition
Binz, M., Dasgupta, I., Jagadish, A. K., Botvinick, M., Wang, J. X., & Schulz, E. (2024) · Behavioral and Brain Sciences · doi:10.1017/s0140525x23003266 · Cited by 38
Linking confidence biases to reinforcement-learning processes.
Salem-Garcia, N., Palminteri, S., & Lebreton, M. (2023) · Psychological Review · doi:10.1037/rev0000424 · Cited by 28
Neural and neurochemical basis of reinforcement-guided decision making
Khani, A., & Rainer, G. (2016) · Journal of Neurophysiology · doi:10.1152/jn.01113.2015 · Cited by 35
Strategies for exploration in the domain of losses
Krueger, P. M., Wilson, R. C., & Cohen, J. D. (2017) · Judgment and Decision Making · doi:10.1017/s1930297500005659 · Cited by 43
Self-citationExpecting the unexpected: a review of learning under uncertainty across development
Topel, S., Ma, I., Sleutels, J. J. M., van Steenbergen, H., de Bruijn, E. R. A., & van Duijvenvoorde, A. C. K. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01098-0 · Cited by 37
Exploitation–Exploration Model of Media Multitasking
Wiradhany, W., Baumgartner, S. E., & de Bruin, A. B. H. (2021) · Journal of Media Psychology Theories Methods and Applications · doi:10.1027/1864-1105/a000303 · Cited by 39
Neurons in human pre-supplementary motor area encode key computations for value-based choice
Aquino, T. G., Cockburn, J., Mamelak, A. N., Rutishauser, U., & O’Doherty, J. P. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01548-2 · Cited by 43
Search for solutions, learning, simulation, and choice processes in suicidal behavior
Dombrovski, A. Y., & Hallquist, M. N. (2021) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1561 · Cited by 41
Interactive Development of Adaptive Learning and Memory
Hartley, C. A., Nussenbaum, K., & Cohen, A. O. (2021) · Annual Review of Developmental Psychology · doi:10.1146/annurev-devpsych-050620-030227 · Cited by 44
Asking better questions: How presentation formats influence information search.
Wu, C. M., Meder, B., Filimon, F., & Nelson, J. D. (2017) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0000374 · Cited by 37
Reward maximization through discrete active inference
Da Costa, L., Sajid, N., Parr, T., Friston, K., & Smith, R. (2023) · Neural Computation · doi:10.1162/neco_a_01574 · Cited by 35
“In most cases, RL implements a simple form of exploration by incorporating randomness in decision making (Tokic & Palm, 2011; Wilson et al., 2014), where the level of randomness may or may not change over time as a function of uncertainty.”
BackgroundOld Strategies, New Environments: Reinforcement Learning on Social Media
Turner, G., Ferguson, A. M., Katiyar, T., Palminteri, S., & Orben, A. C. (2024) · Biological Psychiatry · doi:10.1016/j.biopsych.2024.12.012 · Cited by 36
Computational neuroscience across the lifespan: Promises and pitfalls
van den Bos, W., Bruckner, R., Nassar, M. R., Mata, R., & Eppinger, B. (2017) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2017.09.008 · Cited by 36
Obeso, I., Herrero, M. T., Ligneul, R., Rothwell, J. C., & Jahanshahi, M. (2021) · Neuroscience · doi:10.1016/j.neuroscience.2020.12.035 · Cited by 35
Rational inattention and tonic dopamine
Mikhael, J. G., Lai, L., & Gershman, S. J. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008659 · Cited by 39
The computational relationship between reinforcement learning, social inference, and paranoia
Barnby, J. M., Mehta, M. A., & Moutoussis, M. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010326 · Cited by 38
Empowerment contributes to exploration behaviour in a creative video game
Brändle, F., Stocks, L. J., Tenenbaum, J. B., Gershman, S. J., & Schulz, E. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01661-2 · Cited by 34
Tong, K., Chan, Y. N., Cheng, X., Cheon, B. K., Ellefson, M. R., Fauziana, R., Feng, S., Fischer, N. L., Gulyás, B. Z., Hoo, N. P., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0286208 · Cited by 36
Sources of suboptimality in a minimalistic explore–exploit task
Song, M., Bnaya, Z., & Ma, W. J. (2019) · Nature Human Behaviour · doi:10.1038/s41562-018-0526-x · Cited by 36
Uncertainty in learning, choice, and visual fixation
Stojić, H., Orquin, J. L., Dayan, P., Dolan, R. J., & Speekenbrink, M. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1911348117 · Cited by 40
Loya, M., Sinha, D., & Futrell, R. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.241 · Cited by 33
The dynamics of explore–exploit decisions reveal a signal-to-noise mechanism for random exploration
Feng, S. F., Wang, S., Zarnescu, S., & Wilson, R. C. (2021) · Scientific Reports · doi:10.1038/s41598-021-82530-8 · Cited by 40
Self-citationWolff, W., Radtke, V. C., & Martarelli, C. (2024) · Routledge eBooks · doi:10.4324/9781003271536-3 · Cited by 32
Disturbance modifies payoffs in the explore-exploit trade-off
O’Farrell, S., Sanchirico, J. N., Spiegel, O., Dépalle, M., Haynie, A. C., Murawski, S. A., Perruso, L., & Strelcheck, A. J. (2019) · Nature Communications · doi:10.1038/s41467-019-11106-y · Cited by 36
Smith, R. S., Varshney, L. R., Nagayama, S., Kazama, M., Kitagawa, T., Shunsuke, M., & Ishikawa, Y. (2022) · International Journal of Wellbeing · doi:10.5502/ijw.v12i4.2659 · Cited by 32
Neural responses in macaque prefrontal cortex are linked to strategic exploration
Jahn, C. I., Grohn, J., Cuell, S. F., Emberton, A., Bouret, S., Walton, M. E., Kolling, N., & Sallet, J. (2023) · PLoS Biology · doi:10.1371/journal.pbio.3001985 · Cited by 31
On the Virtues of Fragile Self-Control
Bieleke, M., Wolff, W., & Bertrams, A. (2024) · Routledge eBooks · doi:10.4324/9781003271536-11 · Cited by 32
Optimal utility and probability functions for agents with finite computational precision
Juechems, K., Balaguer, J., Spitzer, B., & Summerfield, C. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2002232118 · Cited by 33
Craig, A. B., Phillips, M. E., Zaldivar, A., Bhattacharyya, R., & Krichmar, J. L. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00017 · Cited by 29
Exploration heuristics decrease during youth
Dubois, M., Bowler, A., Moses-Payne, M. E., Habicht, J., Moran, R., Steinbeis, N., & Hauser, T. U. (2022) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-022-01009-9 · Cited by 31
Chasing Unknown Bandits: Uncertainty Guidance in Learning and Decision Making
Speekenbrink, M. (2022) · Current Directions in Psychological Science · doi:10.1177/09637214221105051 · Cited by 34
Starke, S. D., & Baber, C. (2018) · Applied Ergonomics · doi:10.1016/j.apergo.2018.01.010 · Cited by 29
Mixing It Up: Unsystematic Product Arrangements Promote the Choice of Unfamiliar Products
Walter, M., Hildebrand, C., Häubl, G., & Herrmann, A. P. (2020) · Journal of Marketing Research · doi:10.1177/0022243720901520 · Cited by 30
Differential Effects of Psychotic Illness on Directed and Random Exploration
Waltz, J. A., Wilson, R. C., Albrecht, M. A., Frank, M. J., & Gold, J. M. (2020) · Computational Psychiatry · doi:10.1162/cpsy_a_00027 · Cited by 40
Self-citationLuo, Q., Kanen, J. W., Bari, A., Skandali, N., Langley, C., Knudsen, G. M., Alsiö, J., Phillips, B. U., Sahakian, B. J., Cardinal, R. N., et al. (2023) · Neuropsychopharmacology · doi:10.1038/s41386-023-01762-6 · Cited by 29
Back to reality: differences in learning strategy in a simplified virtual and a real throwing task
Zhang, Z., & Sternad, D. (2020) · Journal of Neurophysiology · doi:10.1152/jn.00197.2020 · Cited by 29
Cameron, C. D., Scheffer, J. A., Hadjiandreou, E., & Anderson, S. (2022) · Advances in experimental social psychology · doi:10.1016/bs.aesp.2022.04.005 · Cited by 35
Neural mechanisms of information seeking
Kobayashi, K., & Kable, J. W. (2024) · Neuron · doi:10.1016/j.neuron.2024.04.008 · Cited by 30
Reinforcement regulates timing variability in thalamus
Wang, J., Hosseini, E. A., Meirhaeghe, N., Akkad, A., & Jazayeri, M. (2020) · eLife · doi:10.7554/elife.55872 · Cited by 27
Parameter and Model Recovery of Reinforcement Learning Models for Restless Bandit Problems
Danwitz, L., Mathar, D., Smith, E. S., Tuzsus, D., & Peters, J. (2022) · Computational Brain & Behavior · doi:10.1007/s42113-022-00139-0 · Cited by 26
Relationships between cognitive biases, decision-making, and delusions
Sheffield, J. M., Smith, R. S., Suthaharan, P., Leptourgos, P., & Corlett, P. R. (2023) · Scientific Reports · doi:10.1038/s41598-023-36526-1 · Cited by 28
Impaired value-based decision-making in Parkinson’s disease apathy
Gilmour, W., Mackenzie, G. M., Feile, M., Tayler-Grint, L., Suveges, S., Macfarlane, J. A., Macleod, A. D., Marshall, V. L., Grunwald, I. Q., Steele, J. D., et al. (2024) · Brain · doi:10.1093/brain/awae025 · Cited by 26
Dynamic noise estimation: A generalized method for modeling noise fluctuations in decision-making
Li, J., Shi, C., Li, L., & Collins, A. G. E. (2024) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2024.102842 · Cited by 29
Inference and Search on Graph-Structured Spaces
Wu, C. M., Schulz, E., & Gershman, S. J. (2020) · Computational Brain & Behavior · doi:10.1007/s42113-020-00091-x · Cited by 29
Marković, D., Goschke, T., & Kiebel, S. J. (2020) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-020-00837-x · Cited by 29
Children perform extensive information gathering when it is not costly
Bowler, A., Habicht, J., Moses-Payne, M. E., Steinbeis, N., Moutoussis, M., & Hauser, T. U. (2020) · Cognition · doi:10.1016/j.cognition.2020.104535 · Cited by 26
Miller, K. J., Botvinick, M., & Brody, C. D. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/461129 · Cited by 30
People adaptively use information to improve their internal states and external outcomes
Dezza, I. C., Maher, C. M., & Sharot, T. (2022) · Cognition · doi:10.1016/j.cognition.2022.105224 · Cited by 30
Multifaceted information-seeking motives in children
Molinaro, G., Dezza, I. C., Bühler, S. K., Moutsiana, C., & Sharot, T. (2023) · Nature Communications · doi:10.1038/s41467-023-40971-x · Cited by 28
Rubin, J. E., Vich, C., Clapp, M., Noneman, K. K., & Verstynen, T. (2020) · European Journal of Neuroscience · doi:10.1111/ejn.14745 · Cited by 28
Heuristics and optimal solutions to the breadth–depth dilemma
Moreno‐Bote, R., Ramírez‐Ruiz, J., Drugowitsch, J., & Hayden, B. Y. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2004929117 · Cited by 31
Controllability boosts neural and cognitive signatures of changes-of-mind in uncertain environments
Rouault, M., Weiss, A., Lee, J. K., Drugowitsch, J., Chambon, V., & Wyart, V. (2022) · eLife · doi:10.7554/elife.75038 · Cited by 26
Deep exploration as a unifying account of explore-exploit behaviorPreprint
Wilson, R. C., Wang, S., Sadeghiyeh, H., & Cohen, J. D. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/uj85c · Cited by 27
Self-citationAdaptive tuning of human learning and choice variability to unexpected uncertainty
Lee, J. K., Rouault, M., & Wyart, V. (2023) · Science Advances · doi:10.1126/sciadv.add0501 · Cited by 26
Ger, Y., Nachmani, E., Wolf, L., & Shahar, N. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011678 · Cited by 26
Behavioural variability contributes to over-staying in patchy foraging
Cash-Padgett, T. V., & Hayden, B. Y. (2020) · Biology Letters · doi:10.1098/rsbl.2019.0915 · Cited by 25
A Multiple-Attribute Decision-Making Approach to Reinforcement Learning
Shi, H., & Xu, M. (2019) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2019.2924724 · Cited by 23
Independent and interacting value systems for reward and information in the human brain
Dezza, I. C., Cleeremans, A., & Alexander, W. H. (2022) · eLife · doi:10.7554/elife.66358 · Cited by 26
The modulation of brain network integration and arousal during exploration
Tardiff, N., Medaglia, J. D., Bassett, D. S., & Thompson‐Schill, S. L. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118369 · Cited by 24
Mice adaptively generate choice variability in a deterministic task
Belkaid, M., Bousseyrol, E., Cuttoli, R. D.-D., Dongelmans, M., Duranté, E. K., Yahia, T. A., Didienne, S., Hanesse, B., Côme, M., Mourot, A., et al. (2020) · Communications Biology · doi:10.1038/s42003-020-0759-x · Cited by 25
Adaptive Decision‐Making “Fast” and “Slow”: A Model of Creative Thinking
Khalil, R., & Brüne, M. (2025) · European Journal of Neuroscience · doi:10.1111/ejn.70024 · Cited by 24
The form of uncertainty affects selection for social learning
Turner, M. A., Moya, C., Smaldino, P. E., & Jones, J. H. (2023) · Evolutionary Human Sciences · doi:10.1017/ehs.2023.11 · Cited by 30
Exploratory preferences explain the human fascination for imaginary worlds in fictional stories
Dubourg, E., Thouzeau, V., de Dampierre, C., Mogoutov, A., & Baumard, N. (2023) · Scientific Reports · doi:10.1038/s41598-023-35151-2 · Cited by 23
Novelty as a drive of human exploration in complex stochastic environments
Modirshanechi, A., Lin, W.-H., He, X., Herzog, M. H., & Gerstner, W. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2502193122 · Cited by 24
Active Inference, Bayesian Optimal Design, and Expected Utility
Sajid, N., Costa, L. D., Parr, T., & Friston, K. J. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009026949.007 · Cited by 27
Pupil Size Encodes Uncertainty during Exploration
Fan, H., Burke, T. D., Sambrano, D. C., Dial, E., Phelps, E. A., & Gershman, S. J. (2023) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_02025 · Cited by 26
Performance errors during rodent learning reflect a dynamic choice strategy
Zhu, Z., & Kuchibhotla, K. V. (2024) · Current Biology · doi:10.1016/j.cub.2024.04.017 · Cited by 21
Temporal trade-offs in psychophysics
Barack, D. L., & Gold, J. I. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.01.015 · Cited by 23
Balancing exploration and exploitation with information and randomizationPreprint
Wilson, R. C., Bonawitz, E. B., Costa, V. D., & Ebitz, R. B. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/e9azw · Cited by 21
Self-citationCombrisson, E., Basanisi, R., Neri, M., Auzias, G., Petri, G., Marinazzo, D., Panzeri, S., & Brovelli, A. (2025) · Nature Communications · doi:10.1038/s41467-025-62507-1 · Cited by 23
Instructed motivational states bias reinforcement learning and memory formation
Sinclair, A. H., Wang, Y. C., & Adcock, R. A. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2304881120 · Cited by 23
Decision-making in brains and robots — the case for an interdisciplinary approach
Lee, S. W., & Seymour, B. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2018.12.012 · Cited by 22
Dynamic Representation of the Subjective Value of Information
Kobayashi, K., Lee, S., Filipowicz, A. L. S., McGaughey, K. D., Kable, J. W., & Nassar, M. R. (2021) · Journal of Neuroscience · doi:10.1523/jneurosci.0423-21.2021 · Cited by 21
Costa, L. D., Sajid, N., Parr, T., Friston, K. J., & Smith, R. S. (2020) · arXiv (Cornell University) · Cited by 23
Dupoux, E., LeCun, Y., & Malik, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.15381 · Cited by 21
A cognitive-computational account of mood swings in adolescence
Gregorová, K., Eldar, E., Deserno, L., & Reiter, A. M. F. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.02.006 · Cited by 21
Bayesian Reinforcement Learning With Limited Cognitive Load
Arumugam, D., Ho, M. K., Goodman, N. D., & Roy, B. V. (2024) · Open Mind · doi:10.1162/opmi_a_00132 · Cited by 28
Simulating News Recommendation Ecosystems for Insights and Implications
Zhang, G., Li, D., Gu, H., Lu, T., Shang, L., & Gu, N. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3381329 · Cited by 20
The influence of anxiety on exploration: A review of computational modeling studies
Chou, K.-P., Wilson, R. C., & Smith, R. S. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105940 · Cited by 20
Self-citationHorváth, L., Colcombe, S. J., Milham, M. P., Ray, S., Schwartenbeck, P., & Ostwald, D. (2021) · Computational Brain & Behavior · doi:10.1007/s42113-021-00112-3 · Cited by 19
Metacognitive Computations for Information Search: Confidence in ControlPreprint
Schulz, L., Fleming, S. M., & Dayan, P. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.03.01.433342 · Cited by 19
Lloyd, A., Roiser, J. P., Skeen, S., Freeman, Z., Badalova, A., Agunbiade, A., Busakhwe, C., DeFlorio, C., Marcu, A., Pirie, H., et al. (2024) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-024-01186-9 · Cited by 17
Robinson, A. H., Chong, T. T. ‐., & Verdejo‐García, A. (2022) · Addiction Biology · doi:10.1111/adb.13172 · Cited by 18
Active inference and the two-step task
Gijsen, S., Grundei, M., & Blankenburg, F. (2022) · Scientific Reports · doi:10.1038/s41598-022-21766-4 · Cited by 18
The structure and development of explore-exploit decision making
Harms, M. B., Xu, Y., Green, C. S., Woodard, K. A., Wilson, R. C., & Pollak, S. D. (2024) · Cognitive Psychology · doi:10.1016/j.cogpsych.2024.101650 · Cited by 24
Self-citationSelective maintenance of value information helps resolve the exploration/exploitation dilemma
Hallquist, M. N., & Dombrovski, A. Y. (2018) · Cognition · doi:10.1016/j.cognition.2018.11.004 · Cited by 19
Brändle, F., Binz, M., & Schulz, E. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009026949.008 · Cited by 18
Metadata
- Authors
- Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A., & Cohen, J. D.
- Affiliations
- Princeton University (US)
- Venue
- Journal of Experimental Psychology: General
- Published
- 2014
- Volume
- 143
- Issue
- 6
- Pages
- 2074-2081
- ISSN
- 0096-3445 (ISSN-L)
- 1939-2222
- PMC
- PMC5635655
- OpenAlex
- W2054698589
- Cited by
- 580
- References
- 34
- Citations by year
- 2026: 52 · 2025: 83 · 2024: 96 · 2023: 67 · 2022: 57 · 2021: 61 · 2020: 53 · 2019: 36 · 2018: 31 · 2017: 22 · 2016: 12 · 2015: 10
- FWCI
- 13.38
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Experimental Psychology General (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- John Templeton Foundation
- J. Insley Blair Pyne
- Topics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Keywords
- exploration-exploitation balance, information seeking, decision noise, random exploration, directed exploration, sequential decision making, time horizon, choice variability, human decision-making
- MeSH terms
- Adolescent, Choice Behavior, Decision Making, Exploratory Behavior, Female, Humans, Male, Neuropsychological Tests, Reward, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.