The DOI matches its Crossref record; 45 of the 46 references it cites were found in a registry or an index.
Deconstructing the human algorithms for exploration
Open accessAbstract
The dilemma between information gathering (exploration) and reward seeking (exploitation) is a fundamental problem for reinforcement learning agents. How humans resolve this dilemma is still an open question, because experiments have provided equivocal evidence about the underlying algorithms used by humans. We show that two families of algorithms can be distinguished in terms of how uncertainty affects exploration. Algorithms based on uncertainty bonuses predict a change in response bias as a function of uncertainty, whereas algorithms based on sampling predict a change in response slope. Two experiments provide evidence for both bias and slope changes, and computational modeling confirms that a hybrid model is the best quantitative account of the data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“UCB exploration bonuses are linked to Bayesian posterior uncertainty: as samples accumulate, the posterior distribution over an option’s value narrows, reducing the exploration bonus20.”
From Interpretable abstractions of artificial neural networks predict behavior and neural activity during human information gathering (2026) · cites this paper as [20]
In the citing paper
“The model has some generalizability: it predicted behavior in a different explore–exploit task20 (Fig. 5b).”
From Interpretable abstractions of artificial neural networks predict behavior and neural activity during human information gathering (2026) · cites this paper as [20]
In the citing paper
“Remarkably, our symbolic functions also outperformed the UCB model in predicting participants’ choices in this different context (Wilcoxon signed-rank test: W = 15.0, n = 89, median difference = −0.07, P = 4.31 × 10−16).”
From Interpretable abstractions of artificial neural networks predict behavior and neural activity during human information gathering (2026) · cites this paper as [20]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
Pattern Recognition and Machine Learning
(2006) · doi:10.1007/978-0-387-45528-0 · Cited by 9,824
Finite-time Analysis of the Multiarmed Bandit Problem
Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002) · Machine Learning · doi:10.1023/a:1013689704352 · Cited by 6,033
THOMPSON, W. R. (1933) · Biometrika · doi:10.1093/biomet/25.3-4.285 · Cited by 2,712
On the Rate of Gain of Information
Hick, W. E. (1952) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470215208416600 · Cited by 2,279
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
Srinivas, N., Krause, A., Kakade, S. M., & Seeger, M. (2012) · IEEE Transactions on Information Theory · doi:10.48550/arxiv.0912.3995 · Cited by 2,100
Stimulus information as a determinant of reaction time.
Hyman, R. (1953) · Journal of Experimental Psychology · doi:10.1037/h0056940 · Cited by 1,443 · cited 1×
“More generally, many studies have found that response time increases with uncertainty (e.g., Gershman, Tenenbaum, & Jäkel, 2016; Grossman, 1953; Hick, 1952; Hyman, 1953).”
BackgroundBayesian model selection for group studies
Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J., & Friston, K. J. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.03.025 · Cited by 1,423
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121
Research works printed but not linked: 1
- Srinivas, N., Krause, A., Seeger, M., & Kakade, S. M. (2010). Gaussian process optimization in the bandit setting: No regret and experimental design. In Proceedings of the 27th international conference on machine learning (pp. 1015–1022).
Cited by
Sorted by citations
Deep reinforcement learning and its neuroscientific implications
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J., & Kurth-Nelson, Z. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.014 · Cited by 269
Computational mechanisms of curiosity and goal-directed exploration
Schwartenbeck, P., Passecker, J., Hauser, T. U., FitzGerald, T. H. B., Kronbichler, M., & Friston, K. J. (2019) · eLife · doi:10.7554/elife.41703 · Cited by 278
Gershman, S. J., & Uchida, N. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0220-7 · Cited by 236
Self-citationWhen Is More Uncertainty Better? A Model of Uncertainty Regulation and Effectiveness
Griffin, M. A., & Grote, G. (2020) · Academy of Management Review · doi:10.5465/amr.2018.0271 · Cited by 223
Balancing exploration and exploitation with information and randomization
Wilson, R. C., Bonawitz, E. B., Costa, V. D., & Ebitz, R. B. (2020) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2020.10.001 · Cited by 236
Computational noise in reward-guided learning drives behavioral variability in volatile environments
Findling, C., Skvortsova, V., Dromnelle, R., Palminteri, S., & Wyart, V. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0518-9 · Cited by 205
The algorithmic architecture of exploration in the human brain
Schulz, E., & Gershman, S. J. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2018.11.003 · Cited by 184
Self-citationSearching for Rewards Like a Child Means Less Generalization and More Directed Exploration
Schulz, E., Wu, C. M., Ruggeri, A., & Meder, B. (2019) · Psychological Science · doi:10.1177/0956797619863663 · Cited by 184
Children are more exploratory and learn more than adults in an approach-avoid task
Liquin, E. G., & Alison Gopnik, C. (2021) · Cognition · doi:10.1016/j.cognition.2021.104940 · Cited by 181
Gershman, S. J. (2018) · Decision · doi:10.1037/dec0000101 · Cited by 146
Self-citationDopaminergic modulation of the exploration/exploitation trade-off in human decision-making
Chakroun, K., Mathar, D., Wiehler, A., Ganzer, F., & Peters, J. (2020) · eLife · doi:10.7554/elife.51260 · Cited by 131
The temporal dynamics of opportunity costs: A normative account of cognitive fatigue and boredom.
Agrawal, M., Mattar, M. G., Cohen, J. D., & Daw, N. D. (2021) · Psychological Review · doi:10.1037/rev0000309 · Cited by 134
Controllability governs the balance between Pavlovian and instrumental action selection
Dorfman, H. M., & Gershman, S. J. (2019) · Nature Communications · doi:10.1038/s41467-019-13737-7 · Cited by 125
Self-citationDevelopment of directed and random exploration in children
Meder, B., Wu, C. M., Schulz, E., & Ruggeri, A. (2021) · Developmental Science · doi:10.1111/desc.13095 · Cited by 130
Deep active inference as variational policy gradients
Millidge, B. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102348 · Cited by 115
Structured, uncertainty-driven exploration in real-world consumer choice
Schulz, E., Bhui, R., Love, B. C., Brier, B., Todd, M. T., & Gershman, S. J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1821028116 · Cited by 111
Self-citationLapses in perceptual decisions reflect exploration
Pisupati, S., Chartarifsky, L., Khanal, A., & Churchland, A. K. (2021) · eLife · doi:10.7554/elife.55490 · Cited by 105
Dissociable neural correlates of uncertainty underlie different exploration strategies
Tomov, M. S., Truong, V. Q., Hundia, R., & Gershman, S. J. (2020) · Nature Communications · doi:10.1038/s41467-020-15766-z · Cited by 102
Self-citationAdaptive Regulation of Motor Variability
Dhawale, A. K., Miyamoto, Y. R., Smith, M. A., & Ölveczky, B. P. (2019) · Current Biology · doi:10.1016/j.cub.2019.08.052 · Cited by 89
Time pressure changes how people explore and respond to uncertainty
Wu, C. M., Schulz, E., Pleskac, T. J., & Speekenbrink, M. (2022) · Scientific Reports · doi:10.1038/s41598-022-07901-1 · Cited by 103
A tutorial on open-source large language models for behavioral science
Hussain, Z., Binz, M., Mata, R., & Wulff, D. U. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02455-8 · Cited by 111
Uncertainty as a driver of the youth mental health crisis
Schweizer, S., Lawson, R. P., & Blakemore, S.-J. (2023) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2023.101657 · Cited by 76
Human complex exploration strategies are enriched by noradrenaline-modulated heuristics
Dubois, M., Habicht, J., Michely, J., Moran, R., Dolan, R. J., & Hauser, T. U. (2021) · eLife · doi:10.7554/elife.59907 · Cited by 76
Computation noise in human learning and decision-making: origin, impact, function
Findling, C., & Wyart, V. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.02.018 · Cited by 83
Cockburn, J., Man, V. Y., Cunningham, W. A., & O’Doherty, J. P. (2022) · Neuron · doi:10.1016/j.neuron.2022.05.025 · Cited by 72
Dubourg, E., & Baumard, N. (2021) · Behavioral and Brain Sciences · doi:10.1017/s0140525x21000923 · Cited by 81
Value-free random exploration is linked to impulsivity
Dubois, M., & Hauser, T. U. (2022) · Nature Communications · doi:10.1038/s41467-022-31918-9 · Cited by 76
Developmental changes in exploration resemble stochastic optimization
Giron, A. P., Ciranka, S., Schulz, E., van den Bos, W., Ruggeri, A., Meder, B., & Wu, C. M. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01662-1 · Cited by 88
Similarities and differences in spatial and non-spatial cognitive maps
Wu, C. M., Schulz, E., Garvert, M. M., Meder, B., & Schuck, N. W. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008149 · Cited by 58
Hierarchical models of pain: Inference, information-seeking, and adaptive control.
Seymour, B., & Mancini, F. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117212 · Cited by 59
Finding structure in multi-armed bandits
Schulz, E., Franklin, N. T., & Gershman, S. J. (2018) · Cognitive Psychology · doi:10.1016/j.cogpsych.2019.101261 · Cited by 57
Self-citationOn the normative advantages of dopamine and striatal opponency for learning and choice
Jaskir, A., & Frank, M. J. (2023) · eLife · doi:10.7554/elife.85107 · Cited by 56
Novelty and uncertainty differentially drive exploration across development
Nussenbaum, K., Martin, R., Maulhardt, S., Yang, Y., Bizzell-Hatcher, G., Bhatt, N. S., König, M., Rosenbaum, G. M., O’Doherty, J. P., Cockburn, J., et al. (2023) · eLife · doi:10.7554/elife.84260 · Cited by 61
Fan, H., Gershman, S. J., & Phelps, E. A. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01455-y · Cited by 66
Self-citationAnnual Research Review: Developmental computational psychiatry
Hauser, T. U., Will, G., Dubois, M., & Dolan, R. J. (2018) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12964 · Cited by 55
It’s new, but is it good? How generalization and uncertainty guide the exploration of novel options.
Stojić, H., Schulz, E., Analytis, P. P., & Speekenbrink, M. (2020) · Journal of Experimental Psychology General · doi:10.1037/xge0000749 · Cited by 57
Smith, R. S., Taylor, S., Wilson, R. C., Chuning, A. E., Persich, M. R., Wang, S., & Killgore, W. D. S. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.782136 · Cited by 56
Dopaminergic genes are associated with both directed and random exploration
Gershman, S. J., & Tzovaras, B. G. (2018) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2018.10.009 · Cited by 48
Self-citationLloyd, A., McKay, R., Sebastian, C. L., & Balsters, J. H. (2021) · Developmental Science · doi:10.1111/desc.13075 · Cited by 49
Curiosity: primate neural circuits for novelty and information seeking
Monosov, I. E. (2024) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00784-9 · Cited by 44
Aberg, K. C., Toren, I., & Paz, R. (2021) · Molecular Psychiatry · doi:10.1038/s41380-021-01363-z · Cited by 48
Olschewski, S., Luckman, A., Mason, A., Ludvig, E. A., & Konstantinidis, E. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231179138 · Cited by 55
Brown, V. M., Hallquist, M. N., Frank, M. J., & Dombrovski, A. Y. (2022) · Cognition · doi:10.1016/j.cognition.2022.105233 · Cited by 51
Meta-learned models of cognition
Binz, M., Dasgupta, I., Jagadish, A. K., Botvinick, M., Wang, J. X., & Schulz, E. (2024) · Behavioral and Brain Sciences · doi:10.1017/s0140525x23003266 · Cited by 38
Temporal discounting correlates with directed exploration but not with random exploration
Sadeghiyeh, H., Wang, S., Alberhasky, M. R., Kyllo, H. M., Shenhav, A., & Wilson, R. C. (2020) · Scientific Reports · doi:10.1038/s41598-020-60576-4 · Cited by 35
Human-AI joint task performance: Learning from uncertainty in autonomous driving systems
Constantinides, P., Monteiro, E., & Mathiassen, L. (2024) · Information and Organization · doi:10.1016/j.infoandorg.2024.100502 · Cited by 39
Travers, E., Friedemann, M., & Haggard, P. (2020) · Cognitive Neuroscience · doi:10.1080/17588928.2020.1824176 · Cited by 42
Xu, T., Singh, K., & Rajivan, P. (2022) · Applied Ergonomics · doi:10.1016/j.apergo.2022.103908 · Cited by 37
Reward maximization through discrete active inference
Da Costa, L., Sajid, N., Parr, T., Friston, K., & Smith, R. (2023) · Neural Computation · doi:10.1162/neco_a_01574 · Cited by 35
Uncertainty in learning, choice, and visual fixation
Stojić, H., Orquin, J. L., Dayan, P., Dolan, R. J., & Speekenbrink, M. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1911348117 · Cited by 34
Neurons in human pre-supplementary motor area encode key computations for value-based choice
Aquino, T. G., Cockburn, J., Mamelak, A. N., Rutishauser, U., & O’Doherty, J. P. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01548-2 · Cited by 37
Reference effects on decision-making elicited by previous rewards
Rigoli, F. (2019) · Cognition · doi:10.1016/j.cognition.2019.104034 · Cited by 33
Searching for rewards like a child means less generalization and more directed explorationPreprint
Schulz, E., Wu, C. M., Ruggeri, A., & Meder, B. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/327593 · Cited by 33
Rational inattention and tonic dopamine
Mikhael, J. G., Lai, L., & Gershman, S. J. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008659 · Cited by 34
Self-citationSmith, R. S., Varshney, L. R., Nagayama, S., Kazama, M., Kitagawa, T., Shunsuke, M., & Ishikawa, Y. (2022) · International Journal of Wellbeing · doi:10.5502/ijw.v12i4.2659 · Cited by 30
Expecting the unexpected: a review of learning under uncertainty across development
Topel, S., Ma, I., Sleutels, J. J. M., van Steenbergen, H., de Bruijn, E. R. A., & van Duijvenvoorde, A. C. K. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01098-0 · Cited by 37
Franklin, N. T., & Frank, M. J. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007720 · Cited by 39
Luo, Q., Kanen, J. W., Bari, A., Skandali, N., Langley, C., Knudsen, G. M., Alsiö, J., Phillips, B. U., Sahakian, B. J., Cardinal, R. N., et al. (2023) · Neuropsychopharmacology · doi:10.1038/s41386-023-01762-6 · Cited by 26
Unifying Principles of Generalization: Past, Present, and Future
Wu, C. M., Meder, B., & Schulz, E. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-021524-110810 · Cited by 39
The dynamics of explore–exploit decisions reveal a signal-to-noise mechanism for random exploration
Feng, S. F., Wang, S., Zarnescu, S., & Wilson, R. C. (2021) · Scientific Reports · doi:10.1038/s41598-021-82530-8 · Cited by 31
Inference and Search on Graph-Structured Spaces
Wu, C. M., Schulz, E., & Gershman, S. J. (2020) · Computational Brain & Behavior · doi:10.1007/s42113-020-00091-x · Cited by 26
Self-citationGlobally Inaccurate Stereotypes Can Result From Locally Adaptive Exploration
Bai, X., Fiske, S. T., & Griffiths, T. L. (2022) · Psychological Science · doi:10.1177/09567976211045929 · Cited by 26
Modernizing Conceptions of Valuation and Cognitive-Control Deployment in Adolescent Risk Taking
Do, K. T., Sharp, P. B., & Telzer, E. H. (2019) · Current Directions in Psychological Science · doi:10.1177/0963721419887361 · Cited by 33
Empowerment contributes to exploration behaviour in a creative video game
Brändle, F., Stocks, L. J., Tenenbaum, J. B., Gershman, S. J., & Schulz, E. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01661-2 · Cited by 34
Self-citationExploration heuristics decrease during youth
Dubois, M., Bowler, A., Moses-Payne, M. E., Habicht, J., Moran, R., Steinbeis, N., & Hauser, T. U. (2022) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-022-01009-9 · Cited by 29
Deep exploration as a unifying account of explore-exploit behaviorPreprint
Wilson, R. C., Wang, S., Sadeghiyeh, H., & Cohen, J. D. (2020) · doi:10.31234/osf.io/uj85c · Cited by 25
The development of creative search strategies
Hart, Y., Kosoy, E., Liquin, E. G., Leonard, J., Mackey, A. P., & Alison Gopnik, C. (2022) · Cognition · doi:10.1016/j.cognition.2022.105102 · Cited by 24
Hertz, U., Bell, V., & Raihani, N. (2021) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2021.1414 · Cited by 24
The modulation of brain network integration and arousal during exploration
Tardiff, N., Medaglia, J. D., Bassett, D. S., & Thompson‐Schill, S. L. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118369 · Cited by 22
A theory of relation learning and cross-domain generalization.
Doumas, L. A. A., Puebla, G., Martin, A. E., & Hummel, J. E. (2022) · Psychological Review · doi:10.1037/rev0000346 · Cited by 31
Chasing Unknown Bandits: Uncertainty Guidance in Learning and Decision Making
Speekenbrink, M. (2022) · Current Directions in Psychological Science · doi:10.1177/09637214221105051 · Cited by 28
Impaired value-based decision-making in Parkinson’s disease apathy
Gilmour, W., Mackenzie, G. M., Feile, M., Tayler-Grint, L., Suveges, S., Macfarlane, J. A., Macleod, A. D., Marshall, V. L., Grunwald, I. Q., Steele, J. D., et al. (2024) · Brain · doi:10.1093/brain/awae025 · Cited by 23
Neural mechanisms of information seeking
Kobayashi, K., & Kable, J. W. (2024) · Neuron · doi:10.1016/j.neuron.2024.04.008 · Cited by 24
Accumbofrontal tract integrity is related to early life adversity and feedback learning
Kennedy, B. V., Hanson, J. L., Buser, N. J., van den Bos, W., Rudolph, K. D., Davidson, R. J., & Pollak, S. D. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01129-9 · Cited by 22
Adaptive tuning of human learning and choice variability to unexpected uncertainty
Lee, J. K., Rouault, M., & Wyart, V. (2023) · Science Advances · doi:10.1126/sciadv.add0501 · Cited by 23
Children perform extensive information gathering when it is not costly
Bowler, A., Habicht, J., Moses-Payne, M. E., Steinbeis, N., Moutoussis, M., & Hauser, T. U. (2020) · Cognition · doi:10.1016/j.cognition.2020.104535 · Cited by 19
Ger, Y., Nachmani, E., Wolf, L., & Shahar, N. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011678 · Cited by 19
Combrisson, E., Basanisi, R., Neri, M., Auzias, G., Petri, G., Marinazzo, D., Panzeri, S., & Brovelli, A. (2025) · Nature Communications · doi:10.1038/s41467-025-62507-1 · Cited by 19
Exploratory preferences explain the human fascination for imaginary worlds in fictional stories
Dubourg, E., Thouzeau, V., de Dampierre, C., Mogoutov, A., & Baumard, N. (2023) · Scientific Reports · doi:10.1038/s41598-023-35151-2 · Cited by 21
Pupil Size Encodes Uncertainty during Exploration
Fan, H., Burke, T. D., Sambrano, D. C., Dial, E., Phelps, E. A., & Gershman, S. J. (2023) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_02025 · Cited by 18
Self-citationFan, H., Gershman, S. J., & Phelps, E. A. (2021) · doi:10.31234/osf.io/yx6sb · Cited by 18
Self-citationUncertainty–guided learning with scaled prediction errors in the basal ganglia
Möller, M., Manohar, S., & Bogacz, R. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009816 · Cited by 16
Selective maintenance of value information helps resolve the exploration/exploitation dilemma
Hallquist, M. N., & Dombrovski, A. Y. (2018) · Cognition · doi:10.1016/j.cognition.2018.11.004 · Cited by 16
Distinct motivations to seek out information in healthy individuals and problem gamblers
Dezza, I. C., Noël, X., Cleeremans, A., & Yu, A. J. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01523-3 · Cited by 16
The influence of anxiety on exploration: A review of computational modeling studies
Chou, K.-P., Wilson, R. C., & Smith, R. S. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105940 · Cited by 18
Simulating News Recommendation Ecosystems for Insights and Implications
Zhang, G., Li, D., Gu, H., Lu, T., Shang, L., & Gu, N. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3381329 · Cited by 18
Behavior Decision of Mobile Robot With a Neurophysiologically Motivated Reinforcement Learning Model
Wang, D., Chen, S., Hu, Y., Liu, L., & Wang, H. (2020) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.3035778 · Cited by 13
The pursuit of happiness: A reinforcement learning perspective on habituation and comparisons
Dubey, R., Griffiths, T. L., & Dayan, P. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010316 · Cited by 18
Lapses in perceptual decisions reflect explorationPreprint
Pisupati, S., Chartarifsky-Lynn, L., Khanal, A., & Churchland, A. K. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/613828 · Cited by 15
Time pressure changes how people explore and respond to uncertaintyPreprint
Wu, C. M., Schulz, E., Pleskac, T. J., & Speekenbrink, M. (2021) · doi:10.31234/osf.io/dsw7q · Cited by 14
Human inference in changing environments with temporal structure.
Prat-Carrabin, A., Wilson, R. C., Cohen, J. D., & Azeredo da Silveira, R. (2021) · Psychological Review · doi:10.1037/rev0000276 · Cited by 16
“Several studies have reported that the responses of human subjects in explore-exploit tasks appeared consistent with Thompson sampling [84, 85, 86].”
ResultThe effects of time horizon and guided choices on explore–exploit decisions in rodents.
Wang, S., Gerken, B., Wieland, J. R., Wilson, R. C., & Fellous, J. (2023) · Behavioral Neuroscience · doi:10.1037/bne0000549 · Cited by 13
A Comparison of Rapid Rule-Learning Strategies in Humans and Monkeys
Goudar, V., Kim, J.-W., Liu, Y., Dede, A. J. O., Jutras, M. J., Skelin, I., Ruvalcaba, M., Chang, W. K., Ram, B., Fairhall, A. L., et al. (2024) · Journal of Neuroscience · doi:10.1523/jneurosci.0231-23.2024 · Cited by 13
Differential impacts of healthy cognitive aging on directed and random exploration.
Mizell, J.-M., Wang, S., Frisvold, A., Alvarado, L., Farrell-Skupny, A., Keung, W., Phelps, C. E., Sundman, M., Franchetti, M.-K., Chou, Y., et al. (2024) · Psychology and Aging · doi:10.1037/pag0000791 · Cited by 14
Developmental changes resemble stochastic optimizationPreprint
Giron, A. P., Ciranka, S., Schulz, E., van den Bos, W., Ruggeri, A., Meder, B., & Wu, C. M. (2022) · doi:10.31234/osf.io/9f4k3 · Cited by 13
Bayesian Reinforcement Learning With Limited Cognitive Load
Arumugam, D., Ho, M. K., Goodman, N. D., & Roy, B. V. (2024) · Open Mind · doi:10.1162/opmi_a_00132 · Cited by 17
Brändle, F., Binz, M., & Schulz, E. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009026949.008 · Cited by 12
Horváth, L., Colcombe, S. J., Milham, M. P., Ray, S., Schwartenbeck, P., & Ostwald, D. (2021) · Computational Brain & Behavior · doi:10.1007/s42113-021-00112-3 · Cited by 12
Compositional Sequence Generation in the Entorhinal–Hippocampal System
McNamee, D., Stachenfeld, K., Botvinick, M., & Gershman, S. J. (2022) · Entropy · doi:10.3390/e24121791 · Cited by 9
Self-citationLukas, D., McCune, K. B., Blaisdell, A. P., Johnson‐Ulrich, Z., MacPherson, M. P., Seitz, B. M., Sevchik, A., & Logan, C. (2024) · Peer Community Journal · doi:10.24072/pcjournal.456 · Cited by 12
Metadata
- Authors
- Gershman, S. J.
- Affiliations
- Harvard University (US)
- Corresponding author
- Gershman, Samuel J.
- ORCID
- Gershman, Samuel J.
- Venue
- Cognition
- Publisher
- Elsevier BV
- Published
- 2018-04
- Volume
- 173
- Pages
- 34-42
- ISSN
- 0010-0277 (ISSN-L)
- PMC
- PMC5801139
- OpenAlex
- W2778689844
- Cited by
- 267
- References
- 46
- Citations by year
- 2026: 38 · 2025: 24 · 2024: 49 · 2023: 35 · 2022: 33 · 2021: 27 · 2020: 29 · 2019: 19 · 2018: 11
- FWCI
- 9.79
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Cognition (acceptedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- NSF Collaborative Research in Computational Neuroscience (IIS-120 7833)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- exploration-exploitation balance, human decision-making, uncertainty, exploration algorithms, response bias, reinforcement learning, computational modeling, hybrid model
- MeSH terms
- Adult, Algorithms, Choice Behavior, Exploratory Behavior, Humans, Psychomotor Performance, Reinforcement, Psychology, Reward, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
