Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts
Open accessAbstract
Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants—even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Higher perceptual rating in the current trial was associated with higher expectation rating in the next trial with a different cue type.
“Interestingly, higher perceptual rating in the current trial was associated with higher expectation rating in the next trial with a different cue type (SI Appendix, Figs. S8–S10) which suggests: 1) the existence of a carryover effect (61–63) from the previous to the current trial independent of the cue, or 2) a general learning of stimulus history.”
From Social information creates self-fulfilling prophecies in judgments of pain, vicarious pain, and cognitive effort (2026) · cites this paper as [63]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879 · see citing works
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524 · see citing works
The assessment and analysis of handedness: The Edinburgh inventory
Oldfield, R. C. (1971) · Neuropsychologia · doi:10.1016/0028-3932(71)90067-4 · Cited by 35,900 · see citing works
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · see citing works
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
A Simplex Method for Function Minimization
Nelder, J. A., & Mead, R. (1965) · The Computer Journal · doi:10.1093/comjnl/7.4.308 · Cited by 28,905 · see citing works
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
A Theory of Cognitive Dissonance
Festinger, L. (1957) · doi:10.1515/9781503620766 · Cited by 24,241 · see citing works
The Ecological Approach to Visual Perception
Bornstein, M. H., & Gibson, J. J. (1980) · The Journal of Aesthetics and Art Criticism · doi:10.2307/429816 · Cited by 22,935 · see citing works
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650 · see citing works
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270 · see citing works
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570 · see citing works
The Ecological Approach to Visual Perception: Classic Edition
Gibson, J. J. (2014) · doi:10.4324/9781315740218 · Cited by 12,132 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · see citing works
Cognitive maps in rats and men
Tolman, E. C. (1948) · Psychological Review · doi:10.1037/h0061626 · Cited by 6,641 · see citing works
Regression and time series model selection in small samples
HURVICH, C. M., & TSAI, C.-L. (1989) · Biometrika · doi:10.1093/biomet/76.2.297 · Cited by 6,586 · see citing works
Barsalou, L. W. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.59.103006.093639 · Cited by 5,936 · see citing works
Rational choice and the structure of the environment.
Simon, H. A. (1956) · Psychological Review · doi:10.1037/h0042769 · Cited by 5,919 · see citing works
Adaptive Mixtures of Local Experts
Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991) · Neural Computation · doi:10.1162/neco.1991.3.1.79 · Cited by 5,384 · see citing works
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · see citing works
The Psychology of Learning and Motivation
(2021) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/s0079-7421(21)x0002-3 · Cited by 5,081 · see citing works
Six views of embodied cognition
Wilson, M. (2002) · Psychonomic Bulletin & Review · doi:10.3758/bf03196322 · Cited by 4,782 · see citing works
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733 · see citing works
Ecological Correlations and the Behavior of Individuals
Robinson, W. S. (1950) · American Sociological Review · doi:10.2307/2087176 · Cited by 4,547 · see citing works
Ratcliff, R. (1978) · Psychological Review · doi:10.1037/0033-295x.85.2.59 · Cited by 4,513 · see citing works
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449 · see citing works
Gigerenzer, G., & Gaissmaier, W. (2011) · Annual Review of Psychology · doi:10.1146/annurev-psych-120709-145346 · Cited by 4,037 · see citing works
Principles of topological psychology.
Lewin, K. (1936) · McGraw-Hill eBooks · doi:10.1037/10019-000 · Cited by 3,967 · see citing works
A DYNAMIC THEORY OF PERSONALITY
Lewin, K. (1936) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-193611000-00051 · Cited by 3,733 · see citing works
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317 · see citing works
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,238 · see citing works
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · see citing works
Hierarchical Mixtures of Experts and the EM Algorithm
Jordan, M. I., & Jacobs, R. A. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.181 · Cited by 2,730 · see citing works
THOMPSON, W. R. (1933) · Biometrika · doi:10.1093/biomet/25.3-4.285 · Cited by 2,712 · see citing works
Individual Choice Behavior: A Theoretical Analysis.
Cane, V. R., & Luce, R. D. (1960) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2343282 · Cited by 2,598 · see citing works
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works
A cortical network for directed attention and unilateral neglect
Mesulam, M. ‐. (1981) · Annals of Neurology · doi:10.1002/ana.410100402 · Cited by 2,498 · see citing works
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321 · see citing works
Repetition and the brain: neural models of stimulus-specific effects
Grill-Spector, K., Henson, R., & Martin, A. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.11.006 · Cited by 2,256 · see citing works
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · see citing works
Self-citationThe time course of perceptual choice: The leaky, competing accumulator model.
Usher, M., & McClelland, J. L. (2001) · Psychological Review · doi:10.1037/0033-295x.108.3.550 · Cited by 2,152 · see citing works
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · see citing works
Self-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053 · see citing works
How are habits formed: modelling habit formation in the real world
Lally, P., van Jaarsveld, C. H. M., Potts, H. W. W., & Wardle, J. (2010) · European Journal of Social Psychology · doi:10.1002/ejsp.674 · Cited by 2,036 · see citing works
Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment.
Busemeyer, J., & Townsend, J. (1993) · Psychology Review · doi:10.1037/0033-295x.100.3.432 · Cited by 2,000 · see citing works
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964 · see citing works
Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning
Yarkoni, T., & Westfall, J. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691617693393 · Cited by 1,930 · see citing works
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910 · see citing works
Normalization as a canonical neural computation
Carandini, M., & Heeger, D. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3136 · Cited by 1,906 · see citing works
Homo Heuristicus: Why Biased Minds Make Better Inferences
Gigerenzer, G., & Brighton, H. (2009) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2008.01006.x · Cited by 1,801 · see citing works
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779 · see citing works
Balleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775 · see citing works
Self-citationPrinciples of Behavior. An Introduction to Behavior Theory
N., E., & Hull, C. L. (1943) · The Journal of Philosophy · doi:10.2307/2019960 · Cited by 1,731 · see citing works
Temporal difference learning and TD-Gammon
Tesauro, G. (1995) · Communications of the ACM · doi:10.1145/203330.203343 · Cited by 1,618 · see citing works
Neural Mechanisms for Interacting with a World Full of Action Choices
Cisek, P., & Kalaska, J. F. (2010) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135409 · Cited by 1,543 · see citing works
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477 · see citing works
Toward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450 · see citing works
Wood, W., & Rünger, D. (2016) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033417 · Cited by 1,445 · see citing works
Visual fixations and the computation and comparison of value in simple choice
Krajbich, I., Armel, C., & Rangel, A. (2010) · Nature Neuroscience · doi:10.1038/nn.2635 · Cited by 1,408 · see citing works
The Basal Ganglia and Adaptive Motor Control
Graybiel, A. M., Aosaki, T., Flaherty, A. W., & Kimura, M. (1994) · Science · doi:10.1126/science.8091209 · Cited by 1,324 · see citing works
Actions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217 · see citing works
Pseudoneglect: a review and meta-analysis of performance factors in line bisection tasks
Jewell, G., & McCourt, M. E. (2000) · Neuropsychologia · doi:10.1016/s0028-3932(99)00045-7 · Cited by 1,187 · see citing works
Sharot, T. (2011) · Current Biology · doi:10.1016/j.cub.2011.10.030 · Cited by 1,187 · see citing works
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166 · see citing works
Self-citationIndividual choice behavior: A theoretical analysis.
Luce, R. D. (2005) · doi:10.1037/14396-000 · Cited by 1,157 · see citing works
Decision making and the avoidance of cognitive demand
Kool, W., McGuire, J. T., Rosen, Z. B., & Botvinick, M. M. (2010) · Journal of Experimental Psychology: General · doi:10.1037/a0020198 · Cited by 1,141 · see citing works
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126 · see citing works
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121 · see citing works
Postdecision changes in the desirability of alternatives.
Brehm, J. W. (1956) · Journal of Abnormal & Social Psychology · doi:10.1037/h0041006 · Cited by 1,109 · see citing works
A lateralized brain network for visuospatial attention
de Schotten, M. T., Dell'Acqua, F., Forkel, S. J., Simmons, A., Vergani, F., Murphy, D. G. M., & Catani, M. (2011) · Nature Neuroscience · doi:10.1038/nn.2905 · Cited by 1,103 · see citing works
Animal intelligence; experimental studies
Thorndike, E. L. (1911) · doi:10.5962/bhl.title.55072 · Cited by 1,100 · see citing works
Cortical mechanisms of action selection: the affordance competition hypothesis
Cisek, P. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2054 · Cited by 1,077 · see citing works
What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior
Behrens, T. E. J., Muller, T. H., Whittington, J. C. R., Mark, S., Baram, A. B., Stachenfeld, K. L., & Kurth-Nelson, Z. (2018) · Neuron · doi:10.1016/j.neuron.2018.10.002 · Cited by 1,055 · see citing works
Neural mechanisms for visual memory and their role in attention
Desimone, R. (1996) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.93.24.13494 · Cited by 1,028 · see citing works
Proceedings of the 35th Annual Conference of the Cognitive Science Society
Dale, R. A. C., Yu, C., Nagai, Y., Coco, M. I., & Kopp, S. (2013) · Cited by 1,006 · see citing works
Right hemisphere dominance for attention
Heilman, K. M., & Abell, T. V. D. (1980) · Neurology · doi:10.1212/wnl.30.3.327 · Cited by 999 · see citing works
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017) · arXiv · doi:10.48550/arxiv.1701.06538 · Cited by 933 · see citing works
Motivation and cognitive control: from behavior to neural mechanism
Botvinick, M., & Braver, T. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010814-015044 · Cited by 909 · see citing works
Reward, Motivation, and Reinforcement Learning
Dayan, P., & Balleine, B. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00963-7 · Cited by 855 · see citing works
Visualizing and Understanding Recurrent Networks
Karpathy, A., Johnson, J., & Fei-Fei, L. (2015) · arXiv · doi:10.48550/arxiv.1506.02078 · Cited by 845 · see citing works
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844 · see citing works
The neurobiological basis of spontaneous alternation
Lalonde, R. (2002) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(01)00041-0 · Cited by 842 · see citing works
Thompson, W. R. (1933) · Biometrika · doi:10.2307/2332286 · Cited by 840 · see citing works
Montague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.11.018 · Cited by 809 · see citing works
Serial dependence in visual perception
Fischer, J., & Whitney, D. C. (2014) · Nature Neuroscience · doi:10.1038/nn.3689 · Cited by 807 · see citing works
Animal intelligence: An experimental study of the associative processes in animals.
Thorndike, E. L. (1898) · The Psychological Monographs · doi:10.1037/h0092987 · Cited by 795 · see citing works
Thorndike, E. L. (1932) · Teachers College Bureau of Publications eBooks · doi:10.1037/10976-000 · Cited by 787 · see citing works
Efron, B., & Morris, C. N. (1977) · Scientific American · doi:10.1038/scientificamerican0577-119 · Cited by 772 · see citing works
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762 · see citing works
Brooks, R. A. (1991) · Science · doi:10.1126/science.253.5025.1227 · Cited by 752 · see citing works
Krajbich, I. M., & Rangel, A. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1101328108 · Cited by 732 · see citing works
How unrealistic optimism is maintained in the face of reality
Sharot, T., Korn, C. W., & Dolan, R. J. (2011) · Nature Neuroscience · doi:10.1038/nn.2949 · Cited by 729 · see citing works
Ten simple rules for the computational modeling of behavioral data
Wilson, R. C., & Collins, A. G. E. (2019) · eLife · doi:10.7554/elife.49547 · Cited by 728 · see citing works
From reinforcement learning models to psychiatric and neurological disorders
Maia, T. V., & Frank, M. J. (2011) · Nature Neuroscience · doi:10.1038/nn.2723 · Cited by 714 · see citing works
Ecological Correlations and the Behavior of Individuals*
Robinson, W. (2009) · International Journal of Epidemiology · doi:10.1093/ije/dyn357 · Cited by 685 · see citing works
Prefrontal cortex and decision making in a mixed-strategy game
Barraclough, D. J., Conroy, M. L., & Lee, D. (2004) · Nature Neuroscience · doi:10.1038/nn1209 · Cited by 678 · see citing works
Interpretable Machine Learning – A Brief History, State-of-the-Art and Challenges
Molnar, C., Casalicchio, G., & Bischl, B. (2020) · Communications in Computer and Information Science · doi:10.1007/978-3-030-65965-3_28 · Cited by 677 · see citing works
Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning
Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0706111104 · Cited by 669 · see citing works
Reinforcement learning, fast and slow
Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.006 · Cited by 667 · see citing works
Disorders of compulsivity: A common bias towards learning habits
Voon, V, Derbyshire, K., Rück, C., Irvine, M. A., Worbe, Y., Enander, J., Schreiber, L. R. N., Gillan, C., Fineberg, N. A., Sahakian, B. J., et al. (2015) · Molecular Psychiatry · doi:10.1038/mp.2014.44 · Cited by 643 · see citing works
Neural computations underlying arbitration between model-based and model-free learning
Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.028 · Cited by 642 · see citing works
Self-citationCognitive effort: A neuroeconomic approach
Westbrook, A., & Braver, T. S. (2015) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-015-0334-y · Cited by 631 · see citing works
Twenty Years of Mixture of Experts
Yuksel, S. E., Wilson, J. N., & Gader, P. D. (2012) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2012.2200299 · Cited by 606 · see citing works
Torres H., J. P. (2025) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.17754506 · Cited by 596 · see citing works
A mathematical model for simple learning.
Bush, R. R., & Mosteller, F. C. (1951) · Psychological Review · doi:10.1037/h0054388 · Cited by 590 · see citing works
Humans use directed and random exploration to solve the explore–exploit dilemma.
Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A., & Cohen, J. D. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0038199 · Cited by 580 · see citing works
Dorsal anterior cingulate cortex and the value of control
Shenhav, A., Cohen, J. D., & Botvinick, M. (2016) · Nature Neuroscience · doi:10.1038/nn.4384 · Cited by 573 · see citing works
How to train your robot with deep reinforcement learning: Lessons we have learned
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920987859 · Cited by 570 · see citing works
Search Dynamics in Consumer Choice under Time Pressure: An Eye-Tracking Study
Reutskaja, E., Nagel, R., Camerer, C. F., & Rangel, A. (2011) · American Economic Review · doi:10.1257/aer.101.2.900 · Cited by 567 · see citing works
Nassar, M. R., Wilson, R. C., Heasly, B. S., & Gold, J. I. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.0822-10.2010 · Cited by 564 · see citing works
Deep Learning of Representations: Looking Forward
Bengio, Y. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39593-2_1 · Cited by 558 · see citing works
The choice axiom after twenty years
Luce, R. D. (1977) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(77)90032-3 · Cited by 555 · see citing works
Westbrook, A., Kester, D., & Braver, T. S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068210 · Cited by 537 · see citing works
Decisions in Changing Conditions: The Urgency-Gating Model
Cisek, P., Puskas, G. A., & El-Murr, S. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.1844-09.2009 · Cited by 521 · see citing works
The Importance of Complexity in Model Selection
Myung, I. J. (2000) · Journal of Mathematical Psychology · doi:10.1006/jmps.1999.1283 · Cited by 519 · see citing works
The limits and potentials of deep learning for robotics
Sünderhauf, N., Brock, O., Scheirer, W. J., Hadsell, R. T., Fox, D., Leitner, J., Upcroft, B., Abbeel, P., Burgard, W., Milford, M. J., et al. (2018) · QUT ePrints (Queensland University of Technology) · Cited by 519 · see citing works
Model‐Based fMRI and Its Application to Reward Learning and Decision Making
O’Doherty, J. P., Hampton, A. N., & Kim, H. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.022 · Cited by 509 · see citing works
Self-citationEvidence for a distributed hierarchy of action representation in the brain
Grafton, S. T., & de C. Hamilton, A. F. (2007) · Human Movement Science · doi:10.1016/j.humov.2007.05.009 · Cited by 503 · see citing works
Self-citationGeneration of random sequences by human subjects: A critical survey of literature.
Wagenaar, W. A. (1972) · Psychological Bulletin · doi:10.1037/h0032060 · Cited by 502 · see citing works
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500 · see citing works
Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1010-06.2006 · Cited by 498 · see citing works
Self-citationLarge-scale analysis of test–retest reliabilities of self-regulation measures
Enkavi, A. Z., Eisenberg, I. W., Bissett, P. G., Mazza, G. L., MacKinnon, D. P., Marsch, L. A., & Poldrack, R. A. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1818430116 · Cited by 489 · see citing works
DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS
Lau, B., & Glimcher, P. W. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.110-04 · Cited by 485 · see citing works
Post-movement beta rebound is generated in motor cortex: Evidence from neuromagnetic recordings
Jurkiewicz, M. T., Gaetz, W. C., Bostan, A. C., & Cheyne, D. O. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.06.005 · Cited by 485 · see citing works
Multiple model-based reinforcement learning
Doya, K., Samejima, K., Katagiri, K.-I., & Kawato, M. (2002) · Neural Computation · doi:10.1162/089976602753712972 · Cited by 476 · see citing works
Asymmetry in the Human Motor Cortex and Handedness
Amunts, K., Gottfried Schlaug, A., Schleicher, A., Steinmetz, H., Dabringhaus, A., Roland, P. E., & Zilles, K. (1996) · NeuroImage · doi:10.1006/nimg.1996.0073 · Cited by 475 · see citing works
Learning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475 · see citing works
Self-citationLearning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements
Амари, Ш.-И. (1972) · IEEE Transactions on Computers · doi:10.1109/t-c.1972.223477 · Cited by 472 · see citing works
Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias
Urai, A. E., Braun, A., & Donner, T. H. (2017) · Nature Communications · doi:10.1038/ncomms14637 · Cited by 466 · see citing works
The striatum: where skills and habits meet
Graybiel, A. M., & Grafton, S. T. (2015) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a021691 · Cited by 463 · see citing works
Shepard, R. N. (1957) · Psychometrika · doi:10.1007/bf02288967 · Cited by 462 · see citing works
Frank, M. J., Doll, B. B., Oas-Terpstra, J., & Moreno, F. (2009) · Nature Neuroscience · doi:10.1038/nn.2342 · Cited by 452 · see citing works
The Importance of Falsification in Computational Cognitive Modeling
Palminteri, S., Wyart, V., & Koechlin, E. (2017) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.03.011 · Cited by 448 · see citing works
Opposite Effects of Recent History on Perception and Decision
Fritsche, M., Mostert, P., & de Lange, F. P. (2017) · Current Biology · doi:10.1016/j.cub.2017.01.006 · Cited by 424 · see citing works
Go and no-go learning in reward and punishment: Interactions between affect and effect
Guitart‐Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Düzel, E., & Dolan, R. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.024 · Cited by 418 · see citing works
Sandson, J. R., & Albert, M. L. (1984) · Neuropsychologia · doi:10.1016/0028-3932(84)90098-8 · Cited by 402 · see citing works
Schönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390 · see citing works
Self-citationGläscher, J., Hampton, A. N., & O'Doherty, J. P. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn098 · Cited by 387 · see citing works
Self-citationThe successor representation in human reinforcement learning
Momennejad, I., Russek, E. M., Cheong, J. H., Botvinick, M., Daw, N. D., & Gershman, S. J. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0180-8 · Cited by 384 · see citing works
Posterior parietal cortex represents sensory history and mediates its effects on behaviour
Akrami, A., Kopec, C. D., Diamond, M. E., & Brody, C. D. (2018) · Nature · doi:10.1038/nature25510 · Cited by 380 · see citing works
Motor sequence learning with the nondominant left hand
Grafton, S. T., Hazeltine, E., & Ivry, R. B. (2002) · Experimental Brain Research · doi:10.1007/s00221-002-1181-y · Cited by 366 · see citing works
Self-citationInterhemispheric asymmetry of the human motor cortex related to handedness and gender
Amunts, K., Jäncke, L., Mohlberg, H., Steinmetz, H., & Zilles, K. (2000) · Neuropsychologia · doi:10.1016/s0028-3932(99)00075-5 · Cited by 365 · see citing works
Differential roles of human striatum and amygdala in associative learning
Li, J., Schiller, D., Schoenbaum, G. M., Phelps, E. A., & Daw, N. D. (2011) · Nature Neuroscience · doi:10.1038/nn.2904 · Cited by 364 · see citing works
Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain
Niv, Y., Edlund, J. A., Dayan, P., & O'Doherty, J. P. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5498-10.2012 · Cited by 356 · see citing works
Self-citationAhn, W., Haines, N., & Zhang, L. (2017) · Computational Psychiatry · doi:10.1162/cpsy_a_00002 · Cited by 354 · see citing works
The Capacity for Generating Information by Randomization
Baddeley, A. (1966) · Quarterly Journal of Experimental Psychology · doi:10.1080/14640746608400019 · Cited by 343 · see citing works
Explainability in deep reinforcement learning
Heuillet, A., Couthouis, F., & Díaz-Rodríguez, N. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106685 · Cited by 337 · see citing works
Adaptive Computation Time for Recurrent Neural Networks
Graves, A. (2016) · arXiv · doi:10.48550/arxiv.1603.08983 · Cited by 335 · see citing works
Hierarchical active inference: A theory of motivated control
Pezzulo, G., Rigoli, F., & Friston, K. J. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.01.009 · Cited by 334 · see citing works
Durkheim's Suicide and Problems of Empirical Research
Selvin, H. C. (1958) · American Journal of Sociology · doi:10.1086/222356 · Cited by 333 · see citing works
A labor/leisure tradeoff in cognitive control.
Kool, W., & Botvinick, M. (2012) · Journal of Experimental Psychology General · doi:10.1037/a0031048 · Cited by 333 · see citing works
Making decisions through a distributed consensus
Cisek, P. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.05.007 · Cited by 329 · see citing works
Dopamine Is Required for the Neural Representation and Control of Movement Vigor
Panigrahi, B., Martin, K. A., Li, Y., Graves, A. R., Vollmer, A., Olson, L., Mensh, B. D., Karpova, A. Y., & Dudman, J. T. (2015) · Cell · doi:10.1016/j.cell.2015.08.014 · Cited by 315 · see citing works
de Ruiter, M. B., Veltman, D. J., Goudriaan, A. E., Oosterlaan, J., Sjoerds, Z., & van den Brink, W. (2008) · Neuropsychopharmacology · doi:10.1038/npp.2008.175 · Cited by 313 · see citing works
Reinforcement learning in artificial and biological systems
Neftci, E., & Averbeck, B. B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0025-4 · Cited by 303 · see citing works
Neural computations underlying action-based decision making in the human brain
Wunderlich, K., Rangel, A., & O'Doherty, J. P. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0901077106 · Cited by 301 · see citing works
Self-citationShepard, R. N. (1958) · Journal of Experimental Psychology · doi:10.1037/h0042354 · Cited by 300 · see citing works
Behavioural and neural characterization of optimistic reinforcement learning
Lefebvre, G., Lebreton, M. P., Meyniel, F., Bourgeois‐Gironde, S., & Palminteri, S. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0067 · Cited by 291 · see citing works
Kool, W., & Botvinick, M. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0401-9 · Cited by 288 · see citing works
Validation of Decision-Making Models and Analysis of Decision Variables in the Rat Basal Ganglia
Ito, M., & Doya, K. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.6157-08.2009 · Cited by 287 · see citing works
The Detection of Visual Contrast in the Behaving Mouse
Busse, L., Ayaz, A., Dhruv, N. T., Katzner, S., Saleem, A. B., Schölvinck, M. L., Zaharia, A. D., & Carandini, M. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.6689-10.2011 · Cited by 286 · see citing works
The convergence of TD(?) for general ?
Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992701 · Cited by 284 · see citing works
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280 · see citing works
Explainable Reinforcement Learning: A Survey
Puiutta, E., & Veith, E. M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57321-8_5 · Cited by 278 · see citing works
Neural correlates of cognitive dissonance and choice-induced preference change
Izuma, K., Matsumoto, M., Murayama, K., Samejima, K., Sadato, N., & Matsumoto, K. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1011879108 · Cited by 277 · see citing works
Review of Deep Reinforcement Learning for Robot Manipulation
Nguyen, V., & La, H. M. (2019) · IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc.2019.00120 · Cited by 275 · see citing works
Deep reinforcement learning and its neuroscientific implications
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J., & Kurth-Nelson, Z. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.014 · Cited by 269 · see citing works
Doing the impossible: A note on induction and the experience of randomness.
Lopes, L. L. (1982) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.8.6.626 · Cited by 268 · see citing works
Deconstructing the human algorithms for exploration
Gershman, S. J. (2018) · Cognition · doi:10.1016/j.cognition.2017.12.014 · Cited by 267 · see citing works
Dissociable Effects of Dopamine and Serotonin on Reversal Learning
den Ouden, H. E. M., Daw, N. D., Fernández, G., Elshout, J. A., Rijpkema, M., Hoogman, M., Franke, B., & Cools, R. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.030 · Cited by 259 · see citing works
Reinforcement Learning in Robotics: Applications and Real-World Challenges
Kormushev, P. S., Calinon, S., & Caldwell, D. G. (2013) · Robotics · doi:10.3390/robotics2030122 · Cited by 259 · see citing works
Rutledge, R. B., Lazzaro, S. C., Lau, B., Myers, C. E., Gluck, M. A., & Glimcher, P. W. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.3524-09.2009 · Cited by 255 · see citing works
Serotonin Selectively Modulates Reward Value in Human Decision-Making
Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., & Dolan, R. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.0053-12.2012 · Cited by 252 · see citing works
The evolutionary psychology of left and right: Costs and benefits of lateralization
Vallortígara, G. (2006) · Developmental Psychobiology · doi:10.1002/dev.20166 · Cited by 252 · see citing works
Influence of reading habits on line bisection
Sylvie Chokron, A., & Imbert, M. (1993) · Cognitive Brain Research · doi:10.1016/0926-6410(93)90005-p · Cited by 248 · see citing works
The production and perception of randomness
Nickerson, R. S. (2002) · Psychological Review · doi:10.1037/0033-295x.109.2.330 · Cited by 247 · see citing works
Modular decomposition in visuomotor learning
Zoubin Ghahramani, A., & Wolpert, D. M. (1997) · Nature · doi:10.1038/386392a0 · Cited by 244 · see citing works
Dorsal hippocampus contributes to model-based planning
Miller, K. J., Botvinick, M., & Brody, C. D. (2017) · Nature Neuroscience · doi:10.1038/nn.4613 · Cited by 243 · see citing works
Procedural learning: II. Intertrial repetition effects in speeded-choice tasks.
Pashler, H., & Baylis, G. C. (1991) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.17.1.33 · Cited by 242 · see citing works
The Artificial Life Route to Artificial Intelligence
Steels, L., & Brooks, R. A. (2018) · Routledge eBooks · doi:10.4324/9781351001885 · Cited by 241 · see citing works
Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems
Kool, W., Gershman, S. J., & Cushman, F. A. (2017) · Psychological Science · doi:10.1177/0956797617708288 · Cited by 240 · see citing works
Recurrent neural networks as versatile tools of neuroscience research
Barak, O. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.06.003 · Cited by 238 · see citing works
Quantifying the effect of intertrial dependence on perceptual decisions
Fründ, I., Wichmann, F. A., & Macke, J. H. (2014) · Journal of Vision · doi:10.1167/14.7.9 · Cited by 237 · see citing works
Sequential Redundancy and Speed in a Serial Two-Choice Responding Task
Bertelson, P. (1961) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470216108416478 · Cited by 236 · see citing works
Animal Intelligence: Experimental Studies
Brown, W. M. (1912) · Nature · doi:10.1038/088306a0 · Cited by 230 · see citing works
Confirmation bias in human reinforcement learning: Evidence from counterfactual feedback processing
Palminteri, S., Lefebvre, G., Kilford, E. J., & Blakemore, S. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005684 · Cited by 230 · see citing works
The anchoring bias reflects rational use of cognitive resources
Lieder, F., Griffiths, T. L., M. Huys, Q. J., & Goodman, N. D. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1286-8 · Cited by 230 · see citing works
Krugel, L. K., Biele, G., Mohr, P. N. C., Li, S.-C., & Heekeren, H. R. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0905191106 · Cited by 229 · see citing works
Mice alternate between discrete strategies during perceptual decision-making
Ashwood, Z. C., Roy, N. A., Stone, I. R., Laboratory, T. I. B., Urai, A. E., Churchland, A. K., Pouget, A., & Pillow, J. W. (2022) · Nature Neuroscience · doi:10.1038/s41593-021-01007-z · Cited by 229 · see citing works
Role of Striatum in Updating Values of Chosen Actions
Kim, H., Sul, J. H., Huh, N., Lee, D., & Jung, M. W. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.2728-09.2009 · Cited by 225 · see citing works
A Review of Sparse Expert Models in Deep LearningPreprint
Fedus, W., Dean, J., & Zoph, B. (2022) · arXiv.org · doi:10.48550/arxiv.2209.01667 · Cited by 220 · see citing works
Expectancy or automatic facilitation? Separating sequential effects in two-choice reaction time.
Soetens, E. L. L., Boer, L. C., & Hueting, J. (1985) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.11.5.598 · Cited by 214 · see citing works
Contributions of the Amygdala to Reward Expectancy and Choice Signals in Human Prefrontal Cortex
Hampton, A. N., Adolphs, R., Tyszka, J. M., & O'Doherty, J. P. (2007) · Neuron · doi:10.1016/j.neuron.2007.07.022 · Cited by 213 · see citing works
Self-citationChoice history biases subsequent evidence accumulation
Urai, A. E., de Gee, J. W., Tsetsos, K., & Donner, T. H. (2019) · eLife · doi:10.7554/elife.46331 · Cited by 212 · see citing works
Flagged first, then by citations
Research works printed but not linked: 10
- [34]Katahira K, Oba T, Toyama A. Can reliability of computational models be truly improved by placing priors on parameters? PsyArXiv. 2022; 4e2d9. 10.31234/osf.io/4e2d9 · DOI
- [135]Banavar NV, Bornstein AM. Multi-plasticities: distinguishing context-specific habits from complex perseverations. In: Vandaele Y, editor. Habits: their definition, neurobiology and role in addiction. Cham, Switzerland: Springer Nature; 2024.
- [178]Greenstreet F, Vergara HM, Pati S, Schwarz L, Wisdom M, Marbach F, Johansson Y, Rollik L, Moskovitz T, Clopath C, Stephenson-Jones M. Action prediction error: a value-free dopaminergic teaching signal that drives stable learning. bioRxiv. 2022; 507572. 10.1101/2022.09.12.507572 · DOI
- [249]Goodfellow I, Bengio Y, Courville A. Deep learning. Cambridge (MA): MIT Press; 2016.
- [277]Tuzsus D, Pappas I, Peters J. Human-level reinforcement learning performance of recurrent neural networks is linked to hyperperseveration, not directed exploration. bioRxiv. 2023; 538570. 10.1101/2023.04.27.538570 · DOI
- [289]Chen Z, Deng Y, Wu Y, Gu Q, Li Y. Towards understanding the mixture-of-experts layer in deep learning. Adv Neural Inf Process Syst. 2022; 35.
- [410]Sutton RS. Temporal credit assignment in reinforcement learning [Doctoral dissertation]. Amherst (MA): University of Massachusetts, Amherst; 1984.
- [414]Watkins CJ. Learning from delayed rewards [Doctoral dissertation]. Cambridge, United Kingdom: University of Cambridge; 1989.
- [416]Rummery GA, Niranjan M. On-line Q-learning using connectionist systems. Cambridge, United Kingdom: Department of Engineering, University of Cambridge; 1994. Technical Report No.: CUED/F-INFENG/TR 166.
- [427]Klopf AH. Brain function and adaptive systems—a heterostatic theory. Bedford (MA): Air Force Cambridge Research Laboratories; 1972. Technical Report No.: AFCRL-72-0164.
Cited by
Sorted by citations
Yazdanpanah, A., Jung, H., Soltani, A., & Wager, T. D. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2513856123 · Cited by 1 · see citing works
“…reward associations with stimulus - action pairs can spill over and bias subsequent policy (61, 82), as well as choice hysteresis, where simply having chosen an option increases the probability of choosing it again on the next trial even when reward contingencies do not warrant it (62, 63, 83, 84).”
- Authors
- Colas, J. T., O’Doherty, J. P., & Grafton, S. T.
- Affiliations
- California Institute of Technology (US)
- University of California, Santa Barbara (US)
- Corresponding author
- Colas, Jaron T.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2024-03-29
- Volume
- 20
- Issue
- 3
- Pages
- e1011950
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC10980507
- OpenAlex
- W4393304821
- Cited by
- 6
- References
- 450
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 2
- FWCI
- 0.88
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Army Research Office (W911NF‑19‑2‑0026, W911NF‑16‑1‑0474)
- National Institute on Drug Abuse (R01 DA040011)
- National Institute of Mental Health (P50 MH094258)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- action bias, mixture of experts, computational modeling of behavior, individual differences in learning, embodied agents, effort cost
- MeSH terms
- Humans, Learning, Reinforcement, Psychology, Reward, Bias, Problem-Based Learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
