The Mixed Instrumental Controller: Using Value of Information to Combine Habitual Choice and Mental Simulation
Open accessAbstract
Instrumental behavior depends on both goal-directed and habitual mechanisms of choice. Normative views cast these mechanisms in terms of model-free and model-based methods of reinforcement learning, respectively. An influential proposal hypothesizes that model-free and model-based mechanisms coexist and compete in the brain according to their relative uncertainty. In this paper we propose a novel view in which a single Mixed Instrumental Controller produces both goal-directed and habitual behavior by flexibly balancing and combining model-based and model-free computations. The Mixed Instrumental Controller performs a cost-benefits analysis to decide whether to chose an action immediately based on the available "cached" value of actions (linked to model-free mechanisms) or to improve value estimation by mentally simulating the expected outcome values (linked to model-based mechanisms). Since mental simulation entails cognitive effort and increases the reward delay, it is activated only when the associated "Value of Information" exceeds its costs. The model proposes a method to compute the Value of Information, based on the uncertainty of action values and on the distance of alternative cached action values. Overall, the model by default chooses on the basis of lighter model-free estimates, and integrates them with costly model-based predictions only when useful. Mental simulation uses a sampling method to produce reward expectancies, which are used to update the cached value of one or more actions; in turn, this updated value is used for the choice. The key predictions of the model are tested in different settings of a double T-maze scenario. Results are discussed in relation with neurobiological evidence on the hippocampus - ventral striatum circuit in rodents, which has been linked to goal-directed spatial navigation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Within this scheme, the balance between goal-directed and habitual components of behaviour can be found in a principled way based on considerations of model accuracy vs. complexity.
“Within this scheme, the balance between goal-directed and habitual components of behaviour can be found in a principled way based on considerations of model model “accuracy” vs. “complexity”; see above and (FitzGerald et al., 2014, Pezzulo et al., 2013).”
From Active Inference, homeostatic regulation and adaptive behavioural control (2015) · cites this paper as Pezzulo et al., 2013
The General Discussion describes an approximate way to balance habitual choice and mental simulation by weighing accuracy of choice against the cognitive effort and time needed to acquire more information. This supports the claimed trade-off, but the paper does not frame it specifically as model accuracy versus model complexity. (LLM, conf=0.78)
Experimental — accuracy not yet measuredSome computational models have adapted a more default interventionist approach in which planning intervenes to alter habits only when necessary.
“More recently, some computational models have adapted a more default interventionist approach in which planning intervenes to alter habits only when necessary (e.g., Pezzulo et al. 2013).”
From Psychology of Habit. (2016) · cites this paper as Pezzulo et al. 2013
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · see citing works
Conflict monitoring and cognitive control
Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001) · Psychological Review · doi:10.1037/0033-295x.108.3.624 · Cited by 7,351 · see citing works
Cognitive maps in rats and men
Tolman, E. C. (1948) · Psychological Review · doi:10.1037/h0061626 · Cited by 6,641 · see citing works
Place navigation impaired in rats with hippocampal lesions
Morris, R. G. M., Garrud, P., Rawlins, J. N. P., & O'Keefe, J. (1982) · Nature · doi:10.1038/297681a0 · Cited by 6,340 · see citing works
The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat
O'Keefe, J., & Dostrovsky, J. (1971) · Brain Research · doi:10.1016/0006-8993(71)90358-1 · Cited by 6,151 · see citing works
Ratcliff, R. (1978) · Psychological Review · doi:10.1037/0033-295x.85.2.59 · Cited by 4,513 · see citing works
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472 · see citing works
On sequential Monte Carlo sampling methods for Bayesian filtering
Doucet, A., Godsill, S., & Andrieu, C. (2000) · Statistics and Computing · doi:10.1023/a:1008935410038 · Cited by 4,466 · see citing works
The Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764 · see citing works
Pattern recognition and machine learning
(2007) · Choice Reviews Online · doi:10.5860/choice.44-5091 · Cited by 2,782 · see citing works
Dynamic bayesian networks: representation, inference and learning
Murphy, K. P., & Russell, S. (2002) · Cited by 2,563 · see citing works
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works
Remembering the past to imagine the future: the prospective brain
Schacter, D. L., Addis, D. R., & Buckner, R. L. (2007) · Nature Reviews Neuroscience · doi:10.1038/nrn2213 · Cited by 2,394 · see citing works
The Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe
Fuster, J. M. (1980) · Cited by 2,367 · see citing works
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · see citing works
Barto, A. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,172 · see citing works
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053 · see citing works
Balleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775 · see citing works
Neural Basis of a Perceptual Decision in the Parietal Cortex (Area LIP) of the Rhesus Monkey
Shadlen, M. N., & Newsome, W. T. (2001) · Journal of Neurophysiology · doi:10.1152/jn.2001.86.4.1916 · Cited by 1,755 · see citing works
Reverse replay of behavioural sequences in hippocampal place cells during the awake state
Foster, D. J., & Wilson, M. A. (2006) · Nature · doi:10.1038/nature04587 · Cited by 1,727 · see citing works
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598 · see citing works
Neural Mechanisms for Interacting with a World Full of Action Choices
Cisek, P., & Kalaska, J. F. (2010) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135409 · Cited by 1,543 · see citing works
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541 · see citing works
Bayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457 · see citing works
The future of memory: remembering, imagining, and the brain
Schacter, D. L., Addis, D. R., Hassabis, D., Martin, V. C., Spreng, R. N., & Szpunar, K. K. (2012) · Neuron · doi:10.1016/j.neuron.2012.11.001 · Cited by 1,398 · see citing works
Sutton, R. S. (1990) · Elsevier eBooks · doi:10.1016/b978-1-55860-141-3.50030-4 · Cited by 1,348 · see citing works
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03095.x · Cited by 1,226 · see citing works
Actions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217 · see citing works
The prefrontal cortex: Anatomy, physiology, and neuropsychology of the frontal lobe
Holmes, G. L. (1989) · Journal of Epilepsy · doi:10.1016/0896-6974(89)90035-2 · Cited by 1,210 · see citing works
Forward and reverse hippocampal place-cell sequences during ripples
Diba, K., & Buzsáki, G. (2007) · Nature Neuroscience · doi:10.1038/nn1961 · Cited by 1,205 · see citing works
The basal ganglia: a vertebrate solution to the selection problem?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Neuroscience · doi:10.1016/s0306-4522(98)00319-4 · Cited by 1,199 · see citing works
An information theoretical approach to prefrontal executive function
Koechlin, E., & Summerfield, C. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.04.005 · Cited by 1,183 · see citing works
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166 · see citing works
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121 · see citing works
Neuroeconomics: decision making and the brain
(2014) · Choice Reviews Online · doi:10.5860/choice.51-5702 · Cited by 1,020 · see citing works
Cisek, P., & Kalaska, J. F. (2005) · Neuron · doi:10.1016/j.neuron.2005.01.027 · Cited by 1,019 · see citing works
Howard, R. A. (1966) · IEEE Transactions on Systems Science and Cybernetics · doi:10.1109/tssc.1966.300074 · Cited by 1,005 · see citing works
Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point
Johnson, A., & Redish, A. D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3761-07.2007 · Cited by 1,003 · see citing works
Medial prefrontal cortex as an action-outcome predictor
Alexander, W. H., & Brown, J. W. (2011) · Nature Neuroscience · doi:10.1038/nn.2921 · Cited by 1,000 · see citing works
Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval
Carr, M. F., Jadhav, S. P., & Frank, L. M. (2011) · Nature Neuroscience · doi:10.1038/nn.2732 · Cited by 958 · see citing works
Neural computations that underlie decisions about sensory stimuli
Gold, J. I., & Shadlen, M. N. (2001) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01567-9 · Cited by 881 · see citing works
Monte-Carlo Planning in Large POMDPs
Silver, D., & Veness, J. (2010) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 838 · see citing works
Building neural representations of habits.
Jog, M. S., Kubota, Y., Connolly, C. I., Hillegaart, V., & Graybiel, A. M. (1999) · Science · doi:10.1126/science.286.5445.1745 · Cited by 834 · see citing works
Bayesian brain : probabilistic approaches to neural coding
賢治, 銅., 信, 石., Pouget, A., & Rao, R. P. N. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/9780262042383.001.0001 · Cited by 823 · see citing works
Prioritized Sweeping: Reinforcement Learning with Less Data and Less Time
Moore, A., & Atkeson, C. G. (1993) · Machine Learning · doi:10.1023/a:1022635613229 · Cited by 823 · see citing works
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809 · see citing works
Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment
Berkes, P., Orbán, G., Lengyel, M., & Fiser, J. (2011) · Science · doi:10.1126/science.1195870 · Cited by 782 · see citing works
Replay of rule-learning related neural patterns in the prefrontal cortex during sleep
Peyrache, A., Khamassi, M., Benchenane, K., Wiener, S. I., & Battaglia, F. P. (2009) · Nature Neuroscience · doi:10.1038/nn.2337 · Cited by 767 · see citing works
The gateway hypothesis of rostral prefrontal cortex (area 10) function
Burgess, P. W., Dumontheil, I., & Gilbert, S. J. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.05.004 · Cited by 753 · see citing works
Stewart, N., Chater, N., & Brown, G. D. A. (2006) · Cognitive Psychology · doi:10.1016/j.cogpsych.2005.10.003 · Cited by 737 · see citing works
Hippocampal replay is not a simple function of experience
Gupta, A. S., van der Meer, M. A. A., Touretzky, D. S., & Redish, A. D. (2010) · Neuron · doi:10.1016/j.neuron.2010.01.034 · Cited by 733 · see citing works
Statistically optimal perception and learning: from behavior to neural representations
Fiser, J., Berkes, P., Orbán, G., & Lengyel, M. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.01.003 · Cited by 672 · see citing works
Neuroeconomics: Decision Making and the Brain
Rushworth, M. F. S., & Walton, M. E. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.005 · Cited by 669 · see citing works
Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective
Botvinick, M. M., Niv, Y., & Barto, A. G. (2009) · Cognition · doi:10.1016/j.cognition.2008.08.011 · Cited by 646 · see citing works
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 636 · see citing works
A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement
Houk, J. C., Adams, J. L., & Barto, A. G. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0020 · Cited by 555 · see citing works
Hierarchical models of behavior and prefrontal function
Botvinick, M. M. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.009 · Cited by 532 · see citing works
Forward frontal fields: phylogeny and fundamental function
Wise, S. P. (2008) · Trends in Neurosciences · doi:10.1016/j.tins.2008.08.008 · Cited by 478 · see citing works
The Role of the Hippocampus in Prediction and Imagination
Buckner, R. L. (2010) · Annual Review of Psychology · doi:10.1146/annurev.psych.60.110707.163508 · Cited by 419 · see citing works
On Sequential Monte Carlo Sampling Methods for Bayesian Filtering
Doucet, A., Godsill, S., & Andrieu, C. (2005) · doi:10.1093/oso/9780199278657.003.0022 · Cited by 416 · see citing works
Yin, H. H., Ostlund, S. B., & Balleine, B. W. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06422.x · Cited by 416 · see citing works
Hippocampus leads ventral striatum in replay of place-reward information
Lansink, C. S., Goltstein, P. M., Lankelma, J. V., McNaughton, B. L., & Pennartz, C. M. A. (2009) · PLoS Biology · doi:10.1371/journal.pbio.1000173 · Cited by 400 · see citing works
The hippocampus: hub of brain network communication for memory
Battaglia, F. P., Benchenane, K., Sirota, A., Pennartz, C. M. A., & Wiener, S. I. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.05.008 · Cited by 351 · see citing works
Speed/accuracy trade-off between the habitual and the goal-directed processes
Keramati, M., Dezfouli, A., & Piray, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002055 · Cited by 346 · see citing works
Cisek, P. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.5605-05.2006 · Cited by 344 · see citing works
Frank, M. J., & Badre, D. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhr114 · Cited by 330 · see citing works
Making decisions through a distributed consensus
Cisek, P. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.05.007 · Cited by 329 · see citing works
Johnson, M. A., Weick, J. P., Pearce, R. A., & Zhang, S.-C. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.4562-06.2007 · Cited by 318 · see citing works
Habits, action sequences and reinforcement learning
Dezfouli, A., & Balleine, B. W. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08050.x · Cited by 297 · see citing works
Covert expectation-of-reward in rat ventral striatum at decision points
van der Meer, M. A. A. (2009) · Frontiers in Integrative Neuroscience · doi:10.3389/neuro.07.001.2009 · Cited by 296 · see citing works
The hippocampal-striatal axis in learning, prediction and goal-directed behavior
Pennartz, C. M. A., Ito, R., Verschure, P. F. M. J., Battaglia, F. P., & Robbins, T. W. (2011) · Trends in Neurosciences · doi:10.1016/j.tins.2011.08.001 · Cited by 287 · see citing works
Caudate Encodes Multiple Computations for Perceptual Decisions
Ding, L., & Gold, J. I. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.2894-10.2010 · Cited by 284 · see citing works
Lateral habenula neurons signal errors in the prediction of reward information
Bromberg-Martin, E. S., & Hikosaka, O. (2011) · Nature Neuroscience · doi:10.1038/nn.2902 · Cited by 275 · see citing works
A normative perspective on motivation
NIV, Y., JOEL, D., & DAYAN, P. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.06.010 · Cited by 251 · see citing works
Retrospective revaluation in sequential decision making: a tale of two systems
Gershman, S. J., Markman, A. B., & Otto, A. R. (2014) · Journal of Experimental Psychology General · doi:10.1037/a0030844 · Cited by 217 · see citing works
Environmentally mediated synergy between perception and behaviour in mobile robots
Verschure, P. F. M. J., Voegtlin, T., & Douglas, R. J. (2003) · Nature · doi:10.1038/nature02024 · Cited by 213 · see citing works
A goal‐directed spatial navigation model using forward trajectory planning based on grid cells
Erdem, U. M., & Hasselmo, M. E. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08015.x · Cited by 209 · see citing works
Rao, R. P. N. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00146 · Cited by 190 · see citing works
Information Processing in Decision-Making Systems
van der Meer, M. A. A., Kurth-Nelson, Z. L., & Redish, A. D. (2012) · The Neuroscientist · doi:10.1177/1073858411435128 · Cited by 186 · see citing works
Neural Correlates of Forward Planning in a Spatial Decision Task in Humans
Simon, D. A., & Daw, N. D. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.4647-10.2011 · Cited by 181 · see citing works
Solway, A., & Botvinick, M. (2012) · Psychological Review · doi:10.1037/a0026435 · Cited by 180 · see citing works
(2007) · MPG.PuRe (Max Planck Society) · Cited by 169 · see citing works
Dopaminergic Modulation of Risky Decision-Making
Simon, N. W., Montgomery, K. S., Beas, B. S., Mitchell, M. R., LaSarge, C. L., Mendez, I. A., Bañuelos, C., Vokes, C. M., Taylor, A. B., Haberman, R. P., et al. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.3772-11.2011 · Cited by 157 · see citing works
Grounding Procedural and Declarative Knowledge in Sensorimotor Anticipation
Pezzulo, G. (2011) · Mind & Language · doi:10.1111/j.1468-0017.2010.01411.x · Cited by 145 · see citing works
Self-citationThinking as the control of imagination: a conceptual framework for goal-directed systems
Pezzulo, G., & Castelfranchi, C. (2009) · Psychological Research Psychologische Forschung · doi:10.1007/s00426-009-0237-z · Cited by 141 · see citing works
Self-citationCoordinating with the Future: The Anticipatory Nature of Representation
Pezzulo, G. (2008) · Minds and Machines · doi:10.1007/s11023-008-9095-5 · Cited by 138 · see citing works
Self-citationIto, M., & Doya, K. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.04.001 · Cited by 131 · see citing works
Multiplicity of control in the basal ganglia: computational roles of striatal subregions
Bornstein, A. M., & Daw, N. D. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.02.009 · Cited by 97 · see citing works
Goal-directed control and its antipodes
Dayan, P. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.004 · Cited by 93 · see citing works
Ventral striatum: a critical look at models of learning and evaluation
van der Meer, M. A. A., & Redish, A. D. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.02.011 · Cited by 84 · see citing works
Model‐based learning and the contribution of the orbitofrontal cortex to the model‐free world
McDannald, M. A., Takahashi, Y. K., Lopatina, N., Pietras, B. W., Jones, J. L., & Schoenbaum, G. M. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07982.x · Cited by 84 · see citing works
Cohen, J. D., McClure, S. M., & Yu, A. J. (2008) · doi:10.1093/oso/9780199230617.003.0014 · Cited by 84 · see citing works
Alterations in choice behavior by manipulations of world model
Green, C. S., Benson, C., Kersten, D., & Schrater, P. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1001709107 · Cited by 83 · see citing works
Control of movements and temporal discounting of reward
Shadmehr, R. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.08.017 · Cited by 75 · see citing works
Neural systems analysis of decision making during goal-directed navigation
Penner, M. R., & Mizumori, S. J. Y. (2011) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2011.08.010 · Cited by 72 · see citing works
Expectancies in decision making, reinforcement learning, and ventral striatum
van der Meer, M. A. A. (2010) · Frontiers in Neuroscience · doi:10.3389/neuro.01.006.2010 · Cited by 67 · see citing works
The Value of Foresight: How Prospection Affects Decision-Making
Pezzulo, G., & Rigoli, F. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00079 · Cited by 66 · see citing works
Self-citationKolter, J. Z., Plagemann, C., Jackson, D. T., Ng, A. Y., & Thrun, S. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509562 · Cited by 62 · see citing works
Chersi, F., Mirolli, M., Pezzulo, G., & Baldassarre, G. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.11.009 · Cited by 55 · see citing works
Self-citationA Bayesian approach to relevance in game playing
Baum, E. B., & Smith, W. D. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00059-3 · Cited by 51 · see citing works
Motor simulation via coupled internal models using sequential Monte Carlo
Dindo, H., Zambuto, D., & Pezzulo, G. (2011) · Nova Science Publishers (Nova Science Publishers, Inc.) · Cited by 51 · see citing works
Self-citationGoal-directed decision making in prefrontal cortex: A computational framework.
Botvinick, M., & An, J. (2009) · PubMed · Cited by 45 · see citing works
The Basal Ganglia Optimize Decision Making over General Perceptual Hypotheses
Lepora, N. F., & GURNEY, K. N. (2012) · Neural Computation · doi:10.1162/neco_a_00360 · Cited by 44 · see citing works
Anticipation and anticipatory behavior
Pezzulo, G., Hoffmann, J., & Falcone, R. (2007) · Cognitive Processing · doi:10.1007/s10339-007-0173-z · Cited by 42 · see citing works
Self-citationPerception, Action, and Utility: The Tangled Skein
Gershman, S. J., & Daw, N. D. (2012) · The MIT Press eBooks · doi:10.7551/mitpress/9108.003.0015 · Cited by 38 · see citing works
O’Doherty, J. P. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08074.x · Cited by 36 · see citing works
Using hippocampal-striatal loops for spatial navigation and goal-directed decision-making
Chersi, F., & Pezzulo, G. (2012) · Cognitive Processing · doi:10.1007/s10339-012-0475-7 · Cited by 33 · see citing works
Self-citationKoene, R. A., & Hasselmo, M. E. (2008) · Neural Networks · doi:10.1016/j.neunet.2007.12.029 · Cited by 30 · see citing works
On the value of information and other rewards
Niv, Y., & Chan, S. C. Y. (2011) · Nature Neuroscience · doi:10.1038/nn.2918 · Cited by 28 · see citing works
Model-Based Reinforcement Learning as Cognitive Search: Neurocomputational Theories
Todd, P. M., Hills, T. T., & Robbins, T. W. (2012) · Cited by 25 · see citing works
Intentional action: from anticipation to goal-directed behavior
Pezzulo, G., & Castelfranchi, C. (2009) · Psychological Research · doi:10.1007/s00426-009-0241-3 · Cited by 25 · see citing works
Self-citationEnvironmental statistics and the trade-off between model-based and TD learning in humans
Simon, D. A., & Daw, N. D. (2011) · neural information processing systems · Cited by 23 · see citing works
Aversive Pavlovian Responses Affect Human Instrumental Motor Performance
Rigoli, F., Pavone, E. F., & Pezzulo, G. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00134 · Cited by 19 · see citing works
Self-citationA neuromorphic model of spatial lookahead planning
Ivey, R., Bullock, D. H., & Grossberg, S. (2010) · Neural Networks · doi:10.1016/j.neunet.2010.11.002 · Cited by 19 · see citing works
Neural Systems Analysis of Decision Making During Goal-Directed Navigation
Mizumori, S. J. Y. (2011) · PsycEXTRA Dataset · doi:10.1037/e597922013-033 · Cited by 6 · see citing works
Anticipation and anticipatory behavior: II
Pezzulo, G., Hoffmann, J., & Falcone, R. (2007) · Cognitive Processing · doi:10.1007/s10339-007-0177-8 · Cited by 3 · see citing works
Self-citationAmbiguity-Reduction: a Satisficing Criterion for Decision Making
Pezzullo, G., & Couyoumdjian, A. (2006) · IRIS Research product catalog (Sapienza University of Rome) · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Wood, W., & Rünger, D. (2016) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033417 · Cited by 1,445 · see citing works
Active inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156 · see citing works
Self-citationDolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977 · see citing works
Active inference and epistemic value
Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., & Pezzulo, G. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1020053 · Cited by 731 · see citing works
Self-citationFriston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., O⿿Doherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674 · see citing works
Self-citationActive Inference, homeostatic regulation and adaptive behavioural control
Pezzulo, G., Rigoli, F., & Friston, K. (2015) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2015.09.001 · Cited by 650 · see citing works
“Although other schemes for hierarchical control have been proposed that use belief precisions or related constructs to modulate the balance of model based and model free controllers (Daw et al., 2005; Gershman and Daw, 2012; Lee et al., 2014; Pezzulo et al., 2013), there are some unique features of the Active Inferenc…”
InfluentialSelf-citationLearning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475 · see citing works
Prioritized memory access explains planning and hippocampal replay
Mattar, M. G., & Daw, N. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0232-z · Cited by 393 · see citing works
Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation
Dayan, P., & Berridge, K. C. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0277-8 · Cited by 357 · see citing works
Predictive representations can link model-based reinforcement learning to model-free mechanisms
Russek, E. M., Momennejad, I., Botvinick, M., Gershman, S. J., & Daw, N. D. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005768 · Cited by 321 · see citing works
Model-based learning protects against forming habits
Gillan, C. M., Otto, A. R., Phelps, E. A., & Daw, N. D. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-015-0347-6 · Cited by 283 · see citing works
Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems
Kool, W., Gershman, S. J., & Cushman, F. A. (2017) · Psychological Science · doi:10.1177/0956797617708288 · Cited by 240 · see citing works
Internally generated sequences in learning and executing goal-directed behavior
Pezzulo, G., van der Meer, M. A. A., Lansink, C. S., & Pennartz, C. M. A. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.06.011 · Cited by 217 · see citing works
Self-citationWhen Does Model-Based Control Pay Off?
Kool, W., Cushman, F. A., & Gershman, S. J. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005090 · Cited by 211 · see citing works
Embodied choice: how action influences perceptual decision making
Lepora, N. F., & Pezzulo, G. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004110 · Cited by 209 · see citing works
Self-citationAdaptive integration of habits into depth-limited planning defines a habitual-goal–directed spectrum
Keramati, M., Smittenaar, P., Dolan, R. J., & Dayan, P. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1609094113 · Cited by 203 · see citing works
The algorithmic anatomy of model-based evaluation
Daw, N. D., & Dayan, P. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0478 · Cited by 195 · see citing works
Deciding How To Decide: Self-Control and Meta-Decision Making
Boureau, Y.-L., Sokol‐Hessner, P., & Daw, N. D. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.08.013 · Cited by 191 · see citing works
Cognitive Control Predicts Use of Model-based Reinforcement Learning
Otto, A. R., Skatova, A., Madlon‐Kay, S., & Daw, N. D. (2014) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00709 · Cited by 186 · see citing works
Mannella, F., Gurney, K., & Baldassarre, G. (2013) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2013.00135 · Cited by 166 · see citing works
Scene Construction, Visual Foraging, and Active Inference
Mirza, M. B., Adams, R. A., Mathys, C. D., & Friston, K. J. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00056 · Cited by 165 · see citing works
Habitual control of goal selection in humans
Cushman, F. A., & Morris, A. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1506367112 · Cited by 165 · see citing works
Prefrontal executive function and adaptive behavior in complex environments
Koechlin, É. (2015) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2015.11.004 · Cited by 154 · see citing works
Algorithms for survival: a comparative perspective on emotions
Bach, D. R., & Dayan, P. (2017) · Nature reviews. Neuroscience · doi:10.1038/nrn.2017.35 · Cited by 144 · see citing works
Akam, T. E., Costa, R. M., & Dayan, P. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004648 · Cited by 150 · see citing works
Integrating memories to guide decisions
Daphna Shohamy, A., & Daw, N. D. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.08.010 · Cited by 141 · see citing works
An information-theoretic perspective on the costs of cognition
Zénon, A., Solopchuk, O., & Pezzulo, G. (2017) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2018.09.013 · Cited by 142 · see citing works
Self-citationFreezing revisited: coordinated autonomic and central optimization of threat coping
Karin Roelofs, A., & Dayan, P. (2022) · Nature reviews. Neuroscience · doi:10.1038/s41583-022-00608-2 · Cited by 141 · see citing works
The why, what, where, when and how of goal-directed choice: neuronal and computational principles
Verschure, P. F. M. J., Pennartz, C. M. A., & Pezzulo, G. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0483 · Cited by 131 · see citing works
Self-citationReinforcement-learning in fronto-striatal circuits
Averbeck, B. B., & O’Doherty, J. P. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01108-0 · Cited by 127 · see citing works
Dorsal anterior cingulate-brainstem ensemble as a reinforcement meta-learner
Silvetti, M., Vassena, E., Abrahamse, E. L., & Verguts, T. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006370 · Cited by 130 · see citing works
Busting Out: Predictive Brains, Embodied Minds, and the Puzzle of the Evidentiary Veil
Clark, A. (2016) · Noûs · doi:10.1111/nous.12140 · Cited by 106 · see citing works
Internally generated hippocampal sequences as a vantage point to probe future‐oriented cognition
Pezzulo, G., Kemere, C. T., & van der Meer, M. A. A. (2017) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13329 · Cited by 93 · see citing works
Self-citationTschantz, A., Barca, L., Maisto, D., Buckley, C. L., Seth, A. K., & Pezzulo, G. (2022) · Biological Psychology · doi:10.1016/j.biopsycho.2022.108266 · Cited by 105 · see citing works
Self-citationThe role of learning-related dopamine signals in addiction vulnerability
Huys, Q. J. M., Tobler, P. N., Hasler, G., & Flagel, S. B. (2014) · Progress in brain research · doi:10.1016/b978-0-444-63425-2.00003-9 · Cited by 87 · see citing works
Hierarchical Action Control: Adaptive Collaboration Between Actions and Habits
Balleine, B. W., & Dezfouli, A. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02735 · Cited by 88 · see citing works
Active inference and robot control: a case study
Pio‐Lopez, L., Nizard, A., Friston, K. J., & Pezzulo, G. (2016) · Journal of The Royal Society Interface · doi:10.1098/rsif.2016.0616 · Cited by 84 · see citing works
Self-citationComputational models of adaptive behavior and prefrontal cortex
Soltani, A., & Koechlin, É. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01123-1 · Cited by 85 · see citing works
Beyond Automaticity: The Psychological Complexity of Skill
Pacherie, É., & Mylopoulos, M. I. (2020) · Topoi · doi:10.1007/s11245-020-09715-0 · Cited by 80 · see citing works
Active Inference, epistemic value, and vicarious trial and errorPreprint
Pezzulo, G., Cartoni, E., Rigoli, F., Pio‐Lopez, L., & Friston, K. J. (2016) · Learning & Memory · doi:10.1101/lm.041780.116 · Cited by 75 · see citing works
Self-citationYoo, A. H., & Collins, A. G. E. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01808 · Cited by 73 · see citing works
Stoianov, I. P., Maisto, D., & Pezzulo, G. (2022) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2022.102329 · Cited by 68 · see citing works
Self-citationForward and Backward Inference in Spatial Cognition
Penny, W. D., Zeidman, P., & Burgess, N. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003383 · Cited by 67 · see citing works
Why and how the brain weights contributions from a mixture of experts
O’Doherty, J. P., Lee, S. W., Tadayonnejad, R., Cockburn, J., Iigaya, K., & Charpentier, C. J. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.10.022 · Cited by 67 · see citing works
The three principles of action: a Pavlovian-instrumental transfer hypothesis
Cartoni, E., Puglisi‐Allegra, S., & Baldassarre, G. (2013) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2013.00153 · Cited by 61 · see citing works
Donnarumma, F., Maisto, D., & Pezzulo, G. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004864 · Cited by 58 · see citing works
Self-citationMaisto, D., Donnarumma, F., & Pezzulo, G. (2015) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.1335 · Cited by 59 · see citing works
Self-citationCompetition and Cooperation Between Multiple Reinforcement Learning Systems
Kool, W., Cushman, F. A., & Gershman, S. J. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-812098-9.00007-3 · Cited by 58 · see citing works
Viejo, G. D., Khamassi, M., Brovelli, A., & Girard, B. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2015.00225 · Cited by 57 · see citing works
Planning Complexity Registers as a Cost in Metacontrol
Kool, W., Gershman, S. J., & Cushman, F. A. (2018) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01263 · Cited by 57 · see citing works
Neural processes mediating contextual influences on human choice behaviour
Rigoli, F., Friston, K. J., & Dolan, R. J. (2016) · Nature Communications · doi:10.1038/ncomms12416 · Cited by 52 · see citing works
Self-citationMaisto, D., Barca, L., den Bergh, O. V., & Pezzulo, G. (2021) · Psychological Review · doi:10.1037/rev0000290 · Cited by 56 · see citing works
Self-citationEpistemic Communities under Active Inference
Albarracin, M., Demekas, D., Ramstead, M. J. D., & Heins, C. (2022) · Entropy · doi:10.3390/e24040476 · Cited by 48 · see citing works
“145 If behavior is initially goal- or information-driven, habit-learning can then ‘zero-in’ and 146 isolate the invariant features of such (initially) goal-directed behavior [75], mirroring the 147 so-called transition from ‘model-based‘ to ‘model-free‘ decision-making in reinforcement 148 learning [76,77].”
BackgroundAn active inference model of hierarchical action understanding, learning and imitation
Proietti, R., Pezzulo, G., & Tessari, A. (2023) · Physics of Life Reviews · doi:10.1016/j.plrev.2023.05.012 · Cited by 47 · see citing works
Self-citationPlanning at decision time and in the background during spatial navigation
Pezzulo, G., Donnarumma, F., Maisto, D., & Stoianov, I. P. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2019.04.009 · Cited by 44 · see citing works
Self-citationCaching mechanisms for habit formation in Active Inference
Maisto, D., Friston, K. J., & Pezzulo, G. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.05.083 · Cited by 46 · see citing works
Self-citationModel-Based Approaches to Active Perception and Control
Pezzulo, G., Donnarumma, F., Iodice, P., Maisto, D., & Stoianov, I. P. (2017) · Entropy · doi:10.3390/e19060266 · Cited by 41 · see citing works
Self-citationPrefrontal Goal Codes Emerge as Latent States in Probabilistic Value Learning
Stoianov, I., Genovesio, A., & Pezzulo, G. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00886 · Cited by 44 · see citing works
Self-citationHumans use forward thinking to exploit social controllability
Na, S., Chung, D., Hula, A., Perl, O., Jung, J., Heflin, M., Blackmore, S., Fiore, V. G., Dayan, P., & Gu, X. (2021) · eLife · doi:10.7554/elife.64983 · Cited by 40 · see citing works
Iodice, P., Ferrante, C., Brunetti, L., Cabib, S., Protasi, F., Walton, M. E., & Pezzulo, G. (2017) · Scientific Reports · doi:10.1038/s41598-017-00561-6 · Cited by 36 · see citing works
Self-citationRealigning Models of Habitual and Goal-Directed Decision-Making
Miller, K. J., Ludvig, E. A., Pezzulo, G., & Shenhav, A. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-812098-9.00018-8 · Cited by 41 · see citing works
Self-citationModel-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysis
Stoianov, I. P., Pennartz, C. M. A., Lansink, C. S., & Pezzulo, G. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006316 · Cited by 37 · see citing works
Self-citationHippocampal replays under the scrutiny of reinforcement learning models
Cazé, R. D., Khamassi, M., Aubin, L., & Girard, B. (2018) · Journal of Neurophysiology · doi:10.1152/jn.00145.2018 · Cited by 38 · see citing works
Active Inference and Learning in the Cerebellum
Friston, K. J., & Herreros, I. (2016) · Neural Computation · doi:10.1162/neco_a_00863 · Cited by 38 · see citing works
Sleep homeostasis, habits and habituation
Vyazovskiy, V. V., Walton, M. E., Peirson, S. N., & Bannerman, D. M. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.05.002 · Cited by 37 · see citing works
From filters to fillers: an active inference approach to body image distortion in the selfie era
Tremblay, S. C., Tremblay, S. E., & Poirier, P. (2020) · AI & Society · doi:10.1007/s00146-020-01015-w · Cited by 38 · see citing works
Understanding, Explanation, and Active Inference
Parr, T., & Pezzulo, G. (2021) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2021.772641 · Cited by 38 · see citing works
Self-citationThe principles of goal-directed decision-making: from neural mechanisms to computation and robotics
Pezzulo, G., Verschure, P. F. M. J., Balkenius, C., & Pennartz, C. M. A. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0470 · Cited by 35 · see citing works
Self-citationBounded Rational Decision-Making from Elementary Computations That Reduce Uncertainty
Gottwald, S., & Braun, D. A. (2019) · Entropy · doi:10.3390/e21040375 · Cited by 35 · see citing works
Tracing the Roots of Cognition in Predictive Processing
Pezzulo, G. (2017) · Theoretical Philosophy/MIND Group – JGU Mainz · doi:10.15502/9783958573215 · Cited by 33 · see citing works
Self-citationCombined model-free and model-sensitive reinforcement learning in non-human primates
Miranda, B., Malalasekera, W. M. N., Behrens, T. E. J., Dayan, P., & Kennerley, S. W. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007944 · Cited by 33 · see citing works
Parallel model-based and model-free reinforcement learning for card sorting performance
Steinke, A., Lange, F., & Kopp, B. (2020) · Scientific Reports · doi:10.1038/s41598-020-72407-7 · Cited by 33 · see citing works
Using hippocampal-striatal loops for spatial navigation and goal-directed decision-making
Chersi, F., & Pezzulo, G. (2012) · Cognitive Processing · doi:10.1007/s10339-012-0475-7 · Cited by 33 · see citing works
Self-citationParallel and hierarchical neural mechanisms for adaptive and predictive behavioral control
Macpherson, T., Matsumoto, M., Gomi, H., Morimoto, J., Uchibe, E., & Hikida, T. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.09.009 · Cited by 30 · see citing works
Analysis of hand kinematics reveals inter-individual differences in intertemporal decision dynamics
Calluso, C., Committeri, G., Pezzulo, G., Lepora, N. F., & Tosoni, A. (2015) · Experimental Brain Research · doi:10.1007/s00221-015-4427-1 · Cited by 31 · see citing works
Self-citationBalancing control: A Bayesian interpretation of habitual and goal-directed behavior
Schwöbel, S., Marković, D., Smolka, M. N., & Kiebel, S. J. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102472 · Cited by 28 · see citing works
Reversal learning strategy in adolescence is associated with prefrontal cortex activation
Boehme, R., Lorenz, R. C., Gleich, T., Romund, L., Pelz, P., Golde, S., Flemming, E., Wold, A., Deserno, L., Behr, J., et al. (2016) · European Journal of Neuroscience · doi:10.1111/ejn.13401 · Cited by 25 · see citing works
Chersi, F., Donnarumma, F., & Pezzulo, G. (2013) · Adaptive Behavior · doi:10.1177/1059712313488789 · Cited by 28 · see citing works
Self-citationDistinct value computations support rapid sequential decisions
Mah, A., Schiereck, S. S., Bossio, V., & Constantinople, C. M. (2023) · Nature Communications · doi:10.1038/s41467-023-43250-x · Cited by 25 · see citing works
Calluso, C., Tosoni, A., Pezzulo, G., Spadone, S., & Committeri, G. (2015) · PLoS ONE · doi:10.1371/journal.pone.0119710 · Cited by 24 · see citing works
Self-citationMiloyan, B. H., Bulley, A., & Suddendorf, T. (2018) · Emotion Review · doi:10.1177/1754073917738570 · Cited by 24 · see citing works
Imaginative Reinforcement Learning: Computational Principles and Neural Mechanisms
Gershman, S. J., Zhou, J., & Kommers, C. (2017) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01170 · Cited by 23 · see citing works
The value of novelty in schizophrenia
Martinelli, C., Rigoli, F., Averbeck, B. B., & Shergill, S. S. (2017) · Schizophrenia Research · doi:10.1016/j.schres.2017.05.007 · Cited by 22 · see citing works
Self-citationModeling awake hippocampal reactivations with model-based bidirectional search
Khamassi, M., & Girard, B. (2020) · Biological Cybernetics · doi:10.1007/s00422-020-00817-x · Cited by 23 · see citing works
Dollé, L., Chavarriaga, R., Guillot, A., & Khamassi, M. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006092 · Cited by 20 · see citing works
Interrupting behaviour: Minimizing decision costs via temporal commitment and low-level interrupts
Lloyd, K., & Dayan, P. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005916 · Cited by 21 · see citing works
Reinforcement learning approaches to hippocampus-dependent flexible spatial navigation
Tessereau, C., O’Dea, R. D., Coombes, S., & Bast, T. (2021) · Brain and Neuroscience Advances · doi:10.1177/2398212820975634 · Cited by 18 · see citing works
Prospective and Pavlovian mechanisms in aversive behaviour
Rigoli, F., Pezzulo, G., & Dolan, R. J. (2015) · Cognition · doi:10.1016/j.cognition.2015.10.017 · Cited by 18 · see citing works
Self-citationQuantifying Motor Task Performance by Bounded Rational Decision Theory
Schach, S., Gottwald, S., & Braun, D. A. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00932 · Cited by 18 · see citing works
Tschantz, A., Barca, L., Maisto, D., Buckley, C. L., Seth, A. K., & Pezzulo, G. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.02.16.431365 · Cited by 18 · see citing works
Self-citationA Goal-Directed Bayesian Framework for Categorization
Rigoli, F., Pezzulo, G., Dolan, R. J., & Friston, K. J. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00408 · Cited by 18 · see citing works
Self-citationComputational approaches to habits in a model-free world
Pauli, W. M., Cockburn, J., Pool, E. R., Perez, O. D., & O’Doherty, J. P. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.12.001 · Cited by 16 · see citing works
Embodied Decisions and the Predictive Brain
Burr, C. (2017) · PhilPapers (PhilPapers Foundation) · doi:10.15502/9783958573086 · Cited by 16 · see citing works
Hotaling, J. M. (2019) · Decision · doi:10.1037/dec0000113 · Cited by 15 · see citing works
Prefrontal dysconnectivity links to working memory deficit in first-episode schizophrenia
Fang, X., Wang, Y., Cheng, L., Zhang, Y., Zhou, Y., Wu, S., Huang, H., Zou, J., Cheng Ann, C. H. E. N., Chen, J., et al. (2017) · Brain Imaging and Behavior · doi:10.1007/s11682-017-9692-0 · Cited by 12 · see citing works
Contextualizing predictive minds
Butz, M. V., Mittenbühler, M., Schwöbel, S., Achimova, A., Gumbsch, C., Otte, S., & Kiebel, S. J. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105948 · Cited by 13 · see citing works
Proietti, R., Parr, T., Tessari, A., Friston, K. J., & Pezzulo, G. (2025) · Physics of Life Reviews · doi:10.1016/j.plrev.2025.05.008 · Cited by 13 · see citing works
Self-citationHabit formation viewed as structural change in the behavioral network
Yamada, K., & Toda, K. (2023) · Communications Biology · doi:10.1038/s42003-023-04500-2 · Cited by 10 · see citing works
The value of uncertainty: An active inference perspective
Pezzulo, G., & Friston, K. J. (2019) · Behavioral and Brain Sciences · doi:10.1017/s0140525x18002066 · Cited by 11 · see citing works
Self-citationO’Reilly, R. C., Nair, A., Russin, J., & Herd, S. A. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00380 · Cited by 11 · see citing works
- Authors
- Pezzulo, G., Rigoli, F., & Chersi, F.
- Affiliations
- Institute for Computational Linguistics “A. Zampolli” (IT)
- National Research Council (IT)
- Corresponding author
- Pezzulo, Giovanni
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2013
- Volume
- 4
- Pages
- 92-92
- ISSN
- 1664-1078 (ISSN-L)
- PMC
- PMC3586710
- OpenAlex
- W2128349062
- Cited by
- 163
- References
- 120
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 9 · 2023: 8 · 2022: 9 · 2021: 17 · 2020: 13 · 2019: 9 · 2018: 14 · 2017: 20 · 2016: 16 · 2015: 17 · 2014: 10 · 2013: 7 · 2012: 1
- FWCI
- 9.14
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Frontiers in Psychology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- City Research Online (City University London) (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- European Commission (270108)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- value of information, model-free reinforcement learning, model-based reinforcement learning, goal-directed behavior, habitual behavior, mental simulation, cost-benefit decision making, reward expectancy, spatial navigation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
