Deconstructing the human algorithms for exploration

Gershman, Samuel J.·2018·Cognition
DOI 10.1016/j.cognition.2017.12.014

Cited by (266)

Items 101–200 of 266

  1. Pallidal neuromodulation of the explore/exploit trade-off in decision-making
    Ana Luísa de Almeida Marcelino, Owen Gray, Bassam Al‐Fatly, William Gilmour, J. Douglas Steele, Andrea A. Kühn, Tom Gilbertson · 2023 · eLife
  2. Impulsivity and risk-seeking as Bayesian inference under dopaminergic control
    John G. Mikhael, Samuel Joseph Gershman · 2021 · Neuropsychopharmacology
  3. The structure and development of explore-exploit decision making
    Madeline B. Harms, Yuyan Xu, C. Shawn Green, Kristina A. Woodard, Robert C. Wilson, Seth D. Pollak · 2024 · Cognitive Psychology
  4. Tabula-rasa exploration decreases during youth and is linked to ADHD symptoms
    Magda Dubois, Aislinn Bowler, Madeleine E. Moses-Payne, Johanna Habicht, Nikolaus Steinbeis, Tobias U. Hauser · 2020 · bioRxiv (Cold Spring Harbor Laboratory)
  5. When should agents explore?
    Mîruna Pislar, David Szepesvári, Georg Ostrovski, Diana Borsa, Tom Schaul · 2021 · arXiv (Cornell University)
  6. Dopamine encoding of novelty facilitates efficient uncertainty-driven exploration
    Yuhao Wang, Armin Lak, Sanjay Manohar, Rafał Bogacz · 2024 · PLoS Computational Biology
  7. Exploration: from machines to humans
    Lior Fox, Ohad Dan, Lotem Elber‐Dorozko, Yonatan Loewenstein · 2020 · Current Opinion in Behavioral Sciences
  8. Motivated Reasoning in an Explore‐Exploit Task
    Zachary A. Caddick, Benjamin Margolin Rottman · 2021 · Cognitive Science
  9. Information-Seeking in the Brain
    Caroline J. Charpentier, Irene Cogliati Dezza · 2022 · Cambridge University Press eBooks
  10. Balancing exploration and exploitation with information and randomization
    Robert C. Wilson, Elizabeth Baraff Bonawitz, Vincent D. Costa, R. Becket Ebitz · 2020 · PsyArXiv (OSF Preprints)
  11. Habits and the subexploration of better transportation options: A dual-system approach
    Bastián Henríquez-Jara, Cristian Ángelo Guevara, Marcela A. Munizaga, Omar David Perez · 2023 · Travel Behaviour and Society
  12. On the use of Wasserstein distance in the distributional analysis of human decision making under uncertainty
    Antonio Candelieri, Andrea Ponti, Ilaria Giordani, Francesco Archetti · 2022 · Annals of Mathematics and Artificial Intelligence
  13. Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T
    Jaron T. Colas, Neil M. Dundon, Raphael T. Gerraty, Natalie M. Saragosa‐Harris, Karol P. Szymula, Koranis Tanwisuth, J. Michael Tyszka, Camilla van Geen, Harang Ju, Arthur W. Toga et al. · 2022 · Human Brain Mapping
  14. Stochastic decisions support optimal foraging of volatile environments, and are disrupted by anxiety
    Alex Lloyd, Ryan McKay, Nicholas Furl A · 2025 · Cognitive Affective & Behavioral Neuroscience
  15. Navigating the Trade-Off between Multi-Task Learning and Learning to Multitask in Deep Neural Networks
    Sachin Ravi, Sebastian Musslick, Maia Hamin, Theodore L. Willke, Jonathan D. Cohen · 2020 · arXiv (Cornell University)
  16. Designing optimal behavioral experiments using machine learning
    Valentin, Simon, Kleinegesse, Steven, Bramley, Neil R, Seriès, Peggy, Gutmann, Michael U, Lucas, Christopher G · 2024 · eLife
  17. Signatures of heuristic-based directed exploration in two-step sequential decision task behaviour
    A.M.A. Brands, David Mathar, Jan Peters · 2025 · Computational Psychiatry
  18. Different exploration strategies along the autism spectrum: diverging effects of autism diagnosis and autism traits
    Fien Goetmaeckers, Judith Goris, Jan Roelf Wiersema, Tom Verguts, Senne Braem · 2025 · Molecular Autism
  19. A causal role of the right dorsolateral prefrontal cortex in random exploration
    Armin Toghi, Mojtaba Chizari, Reza Khosrowabadi · 2024 · Scientific Reports
  20. Individuals with methamphetamine use disorder show reduced directed exploration and learning rates independent of an aversive interoceptive state induction
    Carter M Goldman, Tōru Takahashi, Claire A. Lavalley, Ning Li, Samuel Taylor, Anne E. Chuning, Rowan Hodson, Jennifer Lorraine Stewart, Robert C. Wilson, Sahib S. Khalsa et al. · 2025 · Communications Psychology
  21. From Design Assistants to Design Peers: Turning Daphne into an AI Companion for Mission Designers
    Antoni Virós Martin, Daniel Selva · 2019 · AIAA SciTech Forum
  22. The placebo effect: To explore or to exploit?
    Kirsten Barnes, Benjamin Margolin Rottman, Ben Colagiuri · 2021 · Cognition
  23. A computational model of infant learning and reasoning with probabilities.
    Thomas R. Shultz, Ardavan Salehi Nobandegani · 2021 · Psychological Review
  24. Embracing curiosity eliminates the exploration-exploitation dilemma
    Erik J Peterson, Timothy D Verstynen · 2019 · bioRxiv (Cold Spring Harbor Laboratory)
  25. Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts
    Colas, Jaron T., O’Doherty, John P., Grafton, Scott T. · 2024 · PLOS Computational Biology
  26. Brief mindfulness-based meditation enhances the speed of learning following positive prediction errors
    Marius Golubickis, Lucy Tan, Parnian Jalalian, Johanna K. Falbén, Neil Macrae · 2024 · Quarterly Journal of Experimental Psychology
  27. The neurobehavioral correlates of exploration without learning: Trading off value for explicit, prospective, and variable information gains
    Kristoffer Carl Aberg, Rony Paz · 2024 · Cell Reports
  28. Ambivalence by design: A computational account of loopholes
    Peng Qian, Sophie Bridgers, Maya Taliaferro, Kiera Parece, Tomer David Ullman · 2024 · Cognition
  29. Managerial decision-making: exploration strategies in dynamic environments
    Claire K. Wan, Mingchang Chih · 2024 · Management Decision
  30. The neural correlates of novelty and variability in human decision-making under an active inference framework
    Shuo Zhang, Yan Tian, Quanying Liu, Haiyan Wu · 2024 · eLife
  31. Where Do Heuristics Come From
    Marcel Binz, Dominik Endres · 2019 · eScholarship (California Digital Library)
  32. Investigating the influence of ‘losses disguised as wins’ on decision making and motivation in rats
    Jacqueline‐Marie N. Ferland, Wendy K. Adams, W. Spencer Murch, Linda I. C. Wei, Luke Clark, Catharine A. Winstanley · 2018 · Behavioural Pharmacology
  33. Basal ganglia deep brain stimulation restores cognitive flexibility and exploration-exploitation balance disrupted by NMDA-R antagonism
    Nir Asch, Noa Rahamim, Anna Morozov, Uri Werner-Reiss, Zvi H. Israel, Rony Paz, Hagai Bergman · 2025 · Nature Communications
  34. Rate and noise in human amygdala drive increased exploration in aversive learning
    Tamar Reitich-Stolero, Kristoffer Carl Aberg, Dean Halperin, Carmel Ariel, Genela Morris, Lilach Goldstein, Firas Fahoum, Ido Strauss, Rony Paz · 2025 · Nature
  35. The neural basis for uncertainty processing in hierarchical decision making
    Mien Brabeeba Wang, Nancy Lynch, Michael M. Halassa · 2025 · Nature Communications
  36. Explaining Exploration–Exploitation in Humans
    Antonio Candelieri, Andrea Ponti, Francesco Archetti · 2022 · Big Data and Cognitive Computing
  37. A systematic empirical comparison of active inference and reinforcement learning models in accounting for decision-making under uncertainty
    Ko-Ping Chou, Navid Hakimi, Tzu‐Yu Hsu, Ryan Smith · 2026 · Journal of Mathematical Psychology
  38. Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks
    Kim T. Blackwell, Kenji Doya · 2023 · PLoS Computational Biology
  39. Novelty and uncertainty interact to regulate the balance between exploration and exploitation in the human brain
    Jeffrey Cockburn, Vincent Y. Man, William A. Cunningham, John P. O’Doherty · 2021 · bioRxiv (Cold Spring Harbor Laboratory)
  40. Comparing rapid rule-learning strategies in humans and monkeys
    Vishwa Goudar, Jeong-Woo Kim, Yue Liu, Adam J. O. Dede, Michael J. Jutras, Ivan Skelin, Michael Ruvalcaba, William K. Chang, Adrienne L. Fairhall, Jack J. Lin et al. · 2023 · bioRxiv (Cold Spring Harbor Laboratory)
  41. Model-based exploration is measurable across tasks but not linked to personality and psychiatric assessments
    Kristin Witte, Mirko Thalmann, Eric Schulz · 2025 · Scientific Reports
  42. Revisiting the Role of Uncertainty-Driven Exploration in a (Perceived) Non-Stationary World.
    Dalin Guo, Angela J. Yu · 2021 · PubMed
  43. Softsatisficing: Risk-sensitive softmax action selection
    Takumi Kamiya, Tatsuji Takahashi · 2022 · Biosystems
  44. Learning the Optimal Recommendation from Explorative Users
    Fan Yao, Chuanhao Li, Denis Nekipelov C, Hongning Wang, Haifeng Xu · 2022 · Proceedings of the AAAI Conference on Artificial Intelligence
  45. Search as a simple take-the-best heuristic
    Kyanoush Seyed Yahosseini, Mehdi Moussaïd · 2019 · Royal Society Open Science
  46. Under pressure: The influence of time limits on human exploration
    Charley M. Wu, Eric Schulz, Kimberly Gerbaulet, Timothy Joseph Pleskac, Maarten Speekenbrink · 2019
  47. Human complex exploration strategies are extended via noradrenaline-modulated heuristics
    Magda Dubois, Johanna Habicht, Jochen Michely, Rani Moran, Raymond J. Dolan, Tobias U. Hauser · 2020 · bioRxiv (Cold Spring Harbor Laboratory)
  48. Control Limited Perceptual Decision Making
    Juan R. Castiñeiras de Saa, Alfonso Renart · 2022 · bioRxiv (Cold Spring Harbor Laboratory)
  49. Common roles for serotonin in rats and humans for computations underlying flexible decision-making
    Qiang Luo, Jonathan W. Kanen, Andrea Bari, Nikolina Skandali, Christelle Langley, Gitte M. Knudsen, Johan Alsiö, Benjamin U. Phillips, Barbara Jacquelyn Sahakian, Rudolf N. Cardinal et al. · 2023 · bioRxiv (Cold Spring Harbor Laboratory)
  50. Unifying principles of generalization: past, present, and future
    Charley M. Wu, Björn Meder, Eric Schulz · 2024
  51. Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning
    Sarah Jo C Venditto, Kevin J. Miller, Carlos D. Brody, Nathaniel D. Daw · 2024 · eLife
  52. Mating with Multi-Armed Bandits: Reinforcement Learning Models of Human Mate Search
    Daniel Conroy‐Beam · 2024 · Open Mind
  53. TMS-EEG evidence links random exploration to inhibitory mechanisms in the dorsolateral prefrontal cortex
    Mojtaba Chizari, Keivan Navi, Reza Khosrowabadi · 2025 · Scientific Reports
  54. Exploring and Learning Structure: Active Inference Approach in Navigational Agents
    Daria de Tinguy, Tim Verbelen, Bart Dhoedt · 2024 · Communications in computer and information science
  55. Explore-exploit behavior in humans as a sequential sampling process
    Troy M. Houser · 2025 · Current Psychology
  56. Expecting social punishment facilitates control over a decision under uncertainty by recruiting medial prefrontal cortex
    Jaejoong Kim, Bumseok Jeong · 2020 · Social Cognitive and Affective Neuroscience
  57. How generalization relates to the exploration-exploitation tradeoff
    Troy M. Houser · 2023 · Frontiers in Cognition
  58. Regulation of reinforcement learning parameters captures long‐term changes in rat behaviour
    François Cinotti, Étienne Coutureau, Mehdi Khamassi, Alain R. Marchand, Benoît Girard · 2024 · European Journal of Neuroscience
  59. Why is belief–action consistency so low? The role of belief uncertainty
    Irenaeus Wolff, Dominik Folli · 2024 · Journal of Economic Behavior & Organization
  60. Learning dynamic cognitive map with autonomous navigation
    Daria de Tinguy, Tim Verbelen, Bart Dhoedt · 2024 · Frontiers in Computational Neuroscience
  61. Why so gloomy? A Bayesian explanation of human pessimism bias in the multi-armed bandit task
    Dalin Guo, Angela J. Yu · 2018 · TUbilio (Technical University of Darmstadt)
  62. From Global Optimization to Optimal Learning
    Francesco Archetti, Antonio Candelieri · 2019 · SpringerBriefs in optimization
  63. Thompson Sampling: Predicting Behavior in Games and Markets
    Felix Mauersberger · 2017 · SSRN Electronic Journal
  64. Habit learning supported by efficiently controlled network dynamics in naive macaque monkeys
    Karol P. Szymula, Fabio Pasqualetti, Ann M. Graybiel, Theresa M. Desrochers, Danielle S. Bassett · 2020 · arXiv (Cornell University)
  65. First “Where” and then “How”: Developmental Processes in Exploring Solutions to Problems with Hidden Demands
    Ori Ossmy, Danyang Han, Minxin Cheng, Catherine Bianco, Brianna E. Kaplan, Karen E. Adolph · 2022
  66. Meta-control
    Sebastian Musslick, Jonathan D. Cohen, Thomas Goschke · 2024 · Elsevier eBooks
  67. Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning
    Sarah Jo C Venditto, Kevin J. Miller, Carlos D. Brody, Nathaniel D. Daw · 2024 · eLife
  68. Multiple and subject-specific roles of uncertainty in reward-guided decision-making
    Alexander M. Paunov, Maëva L’Hôtellier, Dalin Guo, Zoe He, Angela J. Yu, Florent Meyniel · 2024 · eLife
  69. Computational Approaches Reveal Developmental Shifts in Exploratory Play
    Joseph A. Colantonio, Ilona Bass, Yee Lee Shing, Sobanawartiny Wijeakumar, Courtney McKay, Eva Rafetseder, Allyson P. Mackey, Elizabeth Baraff Bonawitz · 2025 · Developmental Science
  70. Analyzing Human Search Behavior When Subjective Returns are Unobservable
    Shinji Nakazato, Bojian Yang, Tetsuya Shimokawa · 2023 · Computational Economics
  71. Enactment or Exploration: Two Roles for Philosophy in the Novel of Ideas
    Donald Nordberg · 2023 · Philosophy and literature
  72. Average reward rates enable motivational transfer across independent reinforcement learning tasks
    K. Aberg, R. Paz · 2022 · Frontiers in Behavioral Neuroscience
  73. Relation learning in a neurocomputational architecture supports cross-domain transfer
    Leonidas A. A. Doumas, Guillermo Puebla, Andrea E. Martin, John E. Hummel · 2019 · MPG.PuRe (Max Planck Society)
  74. Distinct motivations to seek out information in healthy and addicted individuals
    Irene Cogliati Dezza, Xavier Noël, Axel Cleeremans, Angela J. Yu · 2020 · bioRxiv (Cold Spring Harbor Laboratory)
  75. A mixture of generative models strategy helps humans generalize across tasks
    Santiago Herce Castañón, Pedro Cardoso-Leite, Irène Altarelli, C. Shawn Green, Paul Schrater, Daphné Bavelier · 2021 · bioRxiv (Cold Spring Harbor Laboratory)
  76. On the computational principles underlying human exploration
    Lior Fox, Ohad Dan, Yonatan Loewenstein · 2023 · eLife
  77. Heuristics for meta-planning from a normative model of information search
    Ionatan Kuperwajs, Mark K. Ho, Wei Ji Ma · 2024 · PsyArXiv (OSF Preprints)
  78. Learning from rewards and social information in naturalistic strategic behavior
    Ionatan Kuperwajs, Bas van Opheusden, Evan M. Russek, Thomas L. Griffiths · 2024
  79. Are ambient smart environments sufficient for developing good habits?
    Valeria Becattini, Liberty Severs · 2025 · Synthese
  80. Increased Baseline Pupil Size Linked to Uncertainty Avoidance in Decision Making
    Ehsan Kakaei, Anne Schlecht, Tobias U. Hauser · 2026 · European Journal of Neuroscience
  81. Interoceptive machine framework: Toward interoception-inspired regulatory architectures in artificial intelligence
    Diego Candia‐Rivera · 2026 · Physics of Life Reviews
  82. Prediction in action: Toward an empirical science of active inference
    Jet Lageman, Johannes J. Fahrenfort, Heleen A. Slagter · 2026 · Neuroscience & Biobehavioral Reviews
  83. The recoverability, reliability, and generalizability of reward processing parameters and relation to mental health symptoms
    Alexandra Claire Pike, Rowan Board, Eoin Travers, Megan Croal, Yeuk Ching Lam, Vincent Valton, Oliver Joe Robinson, Jonathan Paul Roiser · 2026 · Psychological Medicine
  84. Uncertainty reduction as a candidate primary reinforcer: an evolutionary and neural account
    Alan Lincoln · 2026 · Academia Neuroscience and Brain Research
  85. The relationship between temporal discounting and foraging
    Troy M. Houser · 2024 · Current Psychology
  86. Exploration in the wild
    Eric Schulz, Rahul Bhui, Bradley C. Love, Bastien Brier, Michael T. Todd, Samuel Joseph Gershman · 2018 · bioRxiv (Cold Spring Harbor Laboratory)
  87. A neural and behavioral tradeoff underlies exploratory decisions in normative anxiety
    Kristoffer C. Aberg, Ido Toren, Rony Paz · 2020 · bioRxiv (Cold Spring Harbor Laboratory)
  88. Learning from an Exploring Demonstrator: Optimal Reward Estimation for Bandits
    Wenshuo Guo, Kumar Krishna Agrawal, Aditya Grover, Muthukumar, Vidya, Ashwin Pananjady · 2021 · arXiv (Cornell University)
  89. Mixture of Learning Strategies Underlies Rodent Behavior in Dynamic Foraging
    Nhat Minh Le, Murat Yıldırım, Yizhi Wang, 弘記 杉原, Mehrdad Jazayeri, Mriganka Sur · 2022 · bioRxiv (Cold Spring Harbor Laboratory)
  90. On the computational principles underlying human exploration
    Lior Fox, Ohad Dan, Yonatan Loewenstein · 2023
  91. On the computational principles underlying human exploration
    Lior Fox, Ohad Dan, Yonatan Loewenstein · 2023 · eLife
  92. Introducing Dual-System Theory to Travel Behavior: The Relationship between Habits and Sub-Exploration of Novel and Better Transportation Options.
    Bastián Henríquez-Jara, Cristian Ángelo Guevara, Marcela A. Munizaga, Omar David Perez · 2023 · SSRN Electronic Journal
  93. Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning
    Sarah Jo C Venditto, Kevin J. Miller, Carlos D. Brody, Nathaniel D. Daw · 2024 · bioRxiv (Cold Spring Harbor Laboratory)
  94. Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning
    Sarah Jo C Venditto, Kevin J. Miller, Carlos D. Brody, Nathaniel D. Daw · 2024 · eLife
  95. A meta reinforcement learning account of behavioral adaptation to volatility in recurrent neural networks
    Deniz Tuzsus, Ioannis Pappas, Jan Peters · 2024 · bioRxiv (Cold Spring Harbor Laboratory)
  96. Advancing Self-Determination Theory With Computational Intrinsic Motivation: The Case of Competence
    Erik Matias Lintunen, Nadia M. Ady, Christian Guckelsberger B, Sebastian Deterding · 2024
  97. The Neural Correlates of Ambiguity and Risk in Human Decision-Making under an Active Inference Framework
    Shuo Zhang, Yan Tian, Quanying Liu, Haiyan Wu · 2024 · eLife
  98. Multiple and subject-specific roles of uncertainty in reward-guided decision-making
    Alexander M. Paunov, Maëva L’Hôtellier, Dalin Guo, Zoe He, Angela J. Yu, Florent Meyniel · 2024 · eLife
  99. Deep Learning Improves Parameter Estimation in Reinforcement Learning Models
    Hua-Dong Xiong, Jian Li, Marcelo Gomes Mattar, Robert C. Wilson · 2025 · bioRxiv (Cold Spring Harbor Laboratory)
  100. Focused ultrasound neuromodulation of mediodorsal thalamus disrupts decision flexibility during reward learning
    Graeme M Mackenzie, William Gilmour, Isla R. M. Barnard, Shen-Shen Yang, Szabolcs Suveges, Jennifer A. Macfarlane, Avinash Kumar Kanodia, James Manfield, Sadaquate Khan, Jibril Osman‐Farah et al. · 2025 · bioRxiv (Cold Spring Harbor Laboratory)