The DOI matches its Crossref record; 98 of the 98 references it cites were found in a registry or an index.
Computational mechanisms of curiosity and goal-directed exploration
Open accessAbstract
Successful behaviour depends on the right balance between maximising reward and soliciting information about the world. Here, we show how different types of information-gain emerge when casting behaviour as surprise minimisation. We present two distinct mechanisms for goal-directed exploration that express separable profiles of active sampling to reduce uncertainty. ‘Hidden state’ exploration motivates agents to sample unambiguous observations to accurately infer the (hidden) state of the world. Conversely, ‘model parameter’ exploration, compels agents to sample outcomes associated with high uncertainty, if they are informative for their representation of the task structure. We illustrate the emergence of these types of information-gain, termed active inference and active learning, and show how these forms of exploration induce distinct patterns of ‘Bayes-optimal’ behaviour. Our findings provide a computational framework for understanding how distinct levels of uncertainty systematically affect the exploration-exploitation trade-off in decision-making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,414
THOMPSON, W. R. (1933) · Biometrika · doi:10.1093/biomet/25.3-4.285 · Cited by 2,712
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 1×
“In addition, dopamine neurons have been shown to encode ‘infor-mation prediction errors’ analogously to ‘reward prediction errors’ ( Montague et al., 1996 ; Schultz et al., 1997 ).”
BackgroundInformation-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
Srinivas, N., Krause, A., Kakade, S. M., & Seeger, M. W. (2012) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2182033 · Cited by 1,850
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · cited 2×
“These types of uncertainties have been termed unexpected and expected uncertainty ( Yu and Dayan, 2005 ) or, in economics, ambiguity and risk.”
Background(2003) · Cited by 1,680
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598
Attention, uncertainty, and free-energy
Feldman, H., & Friston, K. J. (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00215 · Cited by 1,365
Self-citationUsing confidence bounds for exploitation-exploration trade-offs
Auer, P. L. (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944941 · Cited by 1,229
Bayesian surprise attracts human attention
Itti, L., & Baldi, P. (2009) · Vision Research · doi:10.1016/j.visres.2008.09.007 · Cited by 1,224
Active inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156
Self-citationIntrinsic Motivation Systems for Autonomous Mental Development
Oudeyer, P.-Y., Kaplan, F., & Hafner, V. V. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890271 · Cited by 1,126
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121
Frontal cortex and reward-guided learning and decision-making
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.014 · Cited by 1,111
The Psychology and Neuroscience of Curiosity
Kidd, C., & Hayden, B. Y. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.010 · Cited by 1,026
Howard, R. (1966) · IEEE Transactions on Systems Science and Cybernetics · doi:10.1109/tssc.1966.300074 · Cited by 995
Orbitofrontal Cortex as a Cognitive Map of Task Space
Wilson, R. C., Takahashi, Y. K., Schoenbaum, G., & Niv, Y. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.005 · Cited by 885
Analysis of Thompson Sampling for the multi-armed bandit problem
Agrawal, S., & Goyal, N. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1111.1797 · Cited by 819
What is intrinsic motivation? A typology of computational approaches
Oudeyer, P.-Y. (2007) · Frontiers in Neurorobotics · doi:10.3389/neuro.12.006.2007 · Cited by 811
Information-seeking, curiosity, and attention: computational and neural mechanisms
Gottlieb, J., Oudeyer, P.-Y., Lopes, M., & Baranes, A. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.09.001 · Cited by 784
Kidd, C., Piantadosi, S. T., & Aslin, R. N. (2012) · PLoS ONE · doi:10.1371/journal.pone.0036399 · Cited by 783
(2016) · neural information processing systems · Cited by 735
Active inference and epistemic value
Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., & Pezzulo, G. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1020053 · Cited by 731
Self-citationFriston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., ODoherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674
Self-citationKolling, N., Behrens, T. E. J., Mars, R. B., & Rushworth, M. F. S. (2012) · Science · doi:10.1126/science.1216930 · Cited by 629
Humans use directed and random exploration to solve the explore–exploit dilemma.
Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A., & Cohen, J. D. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0038199 · Cited by 580
Hohwy, J. (2016) · Noûs · doi:10.1111/nous.12062 · Cited by 578
Midbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards
Bromberg-Martin, E. S., & Hikosaka, O. (2009) · Neuron · doi:10.1016/j.neuron.2009.06.009 · Cited by 548
Curious model-building control systems
Schmidhuber, J. (1991) · [Proceedings] 1991 IEEE International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1991.170605 · Cited by 541
Sample mean based index policies by O(log n) regret for the multi-armed bandit problem
Agrawal, R. (1995) · Advances in Applied Probability · doi:10.2307/1427934 · Cited by 540
Exploration by Random Network Distillation
Burda, Y., Edwards, H., Storkey, A., & Klimov, O. (2018) · arXiv · doi:10.48550/arxiv.1810.12894 · Cited by 527 · cited 1×
“This is motivated by a characteristic failure of ‘deep reinforcement learning’ models in tasks that require goal-directed exploratory behaviour, such as the Atari game Montezuma’s revenge ( Burda et al., 2018b ; Mnih et al., 2015 ).”
BackgroundDopamine: generalization and bonuses
Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00048-5 · Cited by 480
VIME: Variational Information Maximizing Exploration
Houthooft, R., Chen, X., Duan, Y., Schulman, J., Turck, F. D., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09674 · Cited by 472
Large-Scale Study of Curiosity-Driven Learning
Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., & Efros, A. A. (2018) · arXiv · doi:10.48550/arxiv.1808.04355 · Cited by 401 · cited 1×
“…based on defining intrinsic motivation as the expected learning progress in a given problem, such that agents ‘plan to be surprised’ ( Barto, 2013 ; Burda et al., 2018a ; Luciw et al., 2013 ; Oudeyer et al., 2007 ; Oudeyer and Kaplan, 2007 ; Schmidhuber, 1991 ; Sun et al., 2011 ; Itti and Baldi,…”
BackgroundDopamine, affordance and active inference
Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., Dolan, R. J., Moran, R., Stephan, K. E., & Bestmann, S. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002327 · Cited by 381 · cited 1×
“In the setting of active inference and learning, the function of dopamine has been linked to the role of the precision of policy selection ( Friston et al., 2014 ; Friston et al., 2012 ).”
MethodSelf-citationCount-Based Exploration with Neural Density Models
Ostrovski, G., Bellemare, M. G., Oord, A. V. D., & Munos, R. (2017) · arXiv · doi:10.48550/arxiv.1703.01310 · Cited by 377 · cited 1×
“Count-based methods use previous visits of a state as a measure for their novelty, with a recent extension of pseudo-count methods to generalise these methods to more complicated (non-tabular) problems ( Bellemare and Srinivasan, 2016 ; Ostrovski and Bellemare, 2017 ;”
MethodActive Inference, Curiosity and Insight
Friston, K. J., Lin, M., Frith, C. D., Pezzulo, G., Hobson, J. A., & Ondobaka, S. (2017) · Neural Computation · doi:10.1162/neco_a_00999 · Cited by 351
Self-citationFrontal cortex subregions play distinct roles in choices between actions and stimuli
Rudebeck, P. H., Behrens, T. E., Kennerley, S. W., Baxter, M. G., Buckley, M. J., Walton, M. E., & Rushworth, M. F. S. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.3541-08.2008 · Cited by 350
Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning
Iglesias, S., Mathys, C., Brodersen, K. H., Kasper, L., Piccirelli, M., den Ouden, H. E. M., & Stephan, K. E. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.009 · Cited by 342
The anatomy of choice: active inference and agency
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00598 · Cited by 340
Self-citationOrbitofrontal Cortex Supports Behavior and Learning Using Inferred But Not Cached Values
Jones, J. L., Esber, G. R., McDannald, M. A., Gruber, A. J., Hernandez, A., Mirenzi, A., & Schoenbaum, G. (2012) · Science · doi:10.1126/science.1227489 · Cited by 335
Hierarchical active inference: A theory of motivated control
Pezzulo, G., Rigoli, F., & Friston, K. J. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.01.009 · Cited by 334 · cited 1×
“Note that in the above parameterisation of an agent’s preferences one would not necessarily distinguish between rewards (or their omission) and punishments, even though one could also introduce these as separate factors over outcomes ( Pezzulo et al., 2018 ).”
BackgroundSelf-citationWaltz, J. A., Frank, M. J., Robinson, B. M., & Gold, J. M. (2007) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.09.042 · Cited by 327
Free Energy, Precision and Learning: The Role of Cholinergic Neuromodulation
Moran, R. J., Campo, P., Symmonds, M., Stephan, K. E., Dolan, R. J., & Friston, K. J. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.4255-12.2013 · Cited by 312
Self-citationRostrolateral Prefrontal Cortex and Individual Differences in Uncertainty-Driven Exploration
Badre, D., Doll, B. B., Long, N. M., & Frank, M. J. (2012) · Neuron · doi:10.1016/j.neuron.2011.12.025 · Cited by 309
Blanchard, T. C., Hayden, B. Y., & Bromberg-Martin, E. S. (2015) · Neuron · doi:10.1016/j.neuron.2014.12.050 · Cited by 295
The anatomy of choice: dopamine and decision-making.
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0481 · Cited by 289
Self-citationTakahashi, Y. K., Roesch, M. R., Stalnaker, T. A., Haney, R. Z., Calu, D. J., Taylor, A. R., Burke, K. A., & Schoenbaum, G. (2009) · Neuron · doi:10.1016/j.neuron.2009.03.005 · Cited by 284
A tutorial on the free-energy framework for modelling perception and learning
Bogacz, R. (2017) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.11.003 · Cited by 283
Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex
Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O'Donnell, P., Niv, Y., & Schoenbaum, G. (2011) · Nature Neuroscience · doi:10.1038/nn.2957 · Cited by 280 · cited 1×
“…of OFC, and it has been hypothesised that the prediction error signal in dopamine neurons may critically rely on OFC input ( Kidd and Hayden, 2015 ; Takahashi et al., 2011 ), and that error-driven learning critically depends on the interplay between those two regions ( Jones et al., 2012 ;…”
BackgroundAn NMR Study of Biomimetic Fluorapatite – Gelatine Mesocrystals
Vyalikh, A., Simon, P., Rosseeva, E., Buder, J., Scheler, U., & Kniep, R. (2015) · Scientific Reports · doi:10.1038/srep15797 · Cited by 275
Deconstructing the human algorithms for exploration
Gershman, S. J. (2018) · Cognition · doi:10.1016/j.cognition.2017.12.014 · Cited by 267
The dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219
Self-citationGrant, R. A., Mitchinson, B., Fox, C. W., & Prescott, T. J. (2009) · Journal of Neurophysiology · doi:10.1152/jn.90783.2008 · Cited by 217
The Goldilocks Effect in Infant Auditory Attention
Kidd, C., Piantadosi, S. T., & Aslin, R. N. (2014) · Child Development · doi:10.1111/cdev.12263 · Cited by 212
Intrinsic Motivation and Reinforcement Learning
Barto, A. G. (2013) · Intrinsically Motivated Learning in Natural and Artificial Systems · doi:10.1007/978-3-642-32375-1_2 · Cited by 211
Dopamine neurons respond to errors in the prediction of sensory features of expected rewards
Takahashi, Y. K., Batchelor, H. M., Liu, B., Khanna, A., Morales, M., & Schoenbaum, G. (2017) · Neuron · doi:10.1016/j.neuron.2017.08.025 · Cited by 200
Computational noise in reward-guided learning drives behavioral variability in volatile environments
Findling, C., Skvortsova, V., Dromnelle, R., Palminteri, S., & Wyart, V. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0518-9 · Cited by 196
The algorithmic architecture of exploration in the human brain
Schulz, E., & Gershman, S. J. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2018.11.003 · Cited by 184
Uncertainty and Exploration in a Restless Bandit Problem
Speekenbrink, M., & Konstantinidis, E. (2015) · Topics in Cognitive Science · doi:10.1111/tops.12145 · Cited by 177
Induction and Relief of Curiosity Elicit Parietal and Frontal Activity
van Lieshout, L. L. F., Vandenbroucke, A. R. E., Müller, N. C. J., Cools, R., & de Lange, F. P. (2018) · Journal of Neuroscience · doi:10.1523/jneurosci.2816-17.2018 · Cited by 169
An information-theoretic approach to curiosity-driven reinforcement learning
Still, S., & Precup, D. (2012) · Theory in Biosciences · doi:10.1007/s12064-011-0142-z · Cited by 165 · cited 1×
“This resolution of uncertainty is closely related to satisfying artificial curiosity ( Schmidhuber, 1991 ; Still and Precup, 2012 ) and the ‘value of information’ ( Howard, 1966 ).”
BackgroundTheoretical perspectives on active sensing
Yang, S. C., Wolpert, D. M., & Lengyel, M. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.06.009 · Cited by 157
Irrational choice and the value of information
Vasconcelos, M., Monteiro, T., & Kacelnik, A. (2015) · Scientific Reports · doi:10.1038/srep13874 · Cited by 152
Separate mesocortical and mesolimbic pathways encode effort and reward learning signals
Hauser, T. U., Eldar, E., & Dolan, R. J. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1705643114 · Cited by 152
Self-citationThe novelty exploration bonus and its attentional modulation☆
Krebs, R. M., Schott, B. H., Schütze, H., & Düzel, E. (2009) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2009.01.015 · Cited by 144
Gershman, S. J. (2018) · Decision · doi:10.1037/dec0000101 · Cited by 142
Planning to be surprised: optimal bayesian exploration in dynamic environments
Sun, Y., Gomez, F., & Schmidhuber, J. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-22887-2_5 · Cited by 134
Morris, S. E., Heerey, E. A., Gold, J. M., & Holroyd, C. B. (2007) · Schizophrenia Research · doi:10.1016/j.schres.2007.08.027 · Cited by 126
The modulation of savouring by prediction error and its effects on choice
Iigaya, K., Story, G. W., Kurth-Nelson, Z. L., Dolan, R. J., & Dayan, P. (2016) · eLife · doi:10.7554/elife.13747 · Cited by 126
Boorman, E. D., Rajendran, V. G., O’Reilly, J. X., & Behrens, T. E. J. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.014 · Cited by 125
Zentall, T. R., & Stagner, J. P. (2010) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2010.1607 · Cited by 124
From relief to surprise: Dual control of epistemic curiosity in the human brain
Ligneul, R., Mermillod, M., & Morisseau, T. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.07.038 · Cited by 120
Wikenheiser, A. M., Marrero-Garcia, Y., & Schoenbaum, G. M. (2017) · Neuron · doi:10.1016/j.neuron.2017.08.003 · Cited by 117
Dopamine, reward learning, and active inference
FitzGerald, T. H. B., Dolan, R. J., & Friston, K. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00136 · Cited by 112 · cited 2×
“Note that we do not address belief updating about precision (QðgÞ) in our simulations, which have been discussed in detail in previous work (FitzGerald et al., 2015; Friston et al., 2017b).”
BackgroundSelf-citationDopaminergic basis for signaling belief updates, but not surprise, and the link to paranoia
Nour, M. M., Dahoun, T., Schwartenbeck, P., Adams, R. A., FitzGerald, T. H. B., Coello, C., Wall, M. B., Dolan, R. J., & Howes, O. D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1809298115 · Cited by 101
Self-citationKoch, K., Schachtzabel, C., Wagner, G., Schikora, J., Schultz, C., Reichenbach, J. R., Sauer, H., & Schlösser, R. G. M. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.031 · Cited by 101 · cited 1×
“For example, individuals diagnosed with Schizophrenia and schizoaffective disorders are reported to suffer from cognitive deficits associated with both active inference and active learning ( Koch et al., 2010 ; Morris et al., 2008 ; Waltz et al., 2007 ; Weickert et al., 2009 ).”
BackgroundPure correlates of exploration and exploitation in the human brain
Blanchard, T. C., & Gershman, S. J. (2017) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-017-0556-2 · Cited by 96
EX2: Exploration with Exemplar Models for Deep Reinforcement Learning
Fu, J., Co-Reyes, J. D., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.01260 · Cited by 95
Multiple Modes of Phase Locking between Sniffing and Whisking during Active Exploration
Ranade, S., Hangya, B., & Kepecs, Á. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.3874-12.2013 · Cited by 93
Neural signals encoding shifts in beliefs
Schwartenbeck, P., FitzGerald, T. H. B., & Dolan, R. J. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.10.067 · Cited by 92
Self-citationNeural Correlates of Probabilistic Category Learning in Patients with Schizophrenia
Weickert, T. W., Goldberg, T. E., Callicott, J. H., Chen, Q., Apud, J. A., Das, S., Zoltick, B. J., Egan, M. F., Meeter, M., Myers, C. E., et al. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.4341-08.2009 · Cited by 90
Curiosity Driven Exploration of Learned Disentangled Goal SpacesPreprint
Laversanne-Finot, A., Péré, A., & Oudeyer, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.01521 · Cited by 87
Prediction of primary somatosensory neuron activity during active tactile exploration
Campagner, D., Evans, M. H., Bale, M. R., Erskine, A., & Petersen, R. S. (2016) · eLife · doi:10.7554/elife.10696 · Cited by 82
Control of entropy in neural models of environmental state
Müller, T., Mars, R. B., Behrens, T. E. J., & O’Reilly, J. X. (2019) · eLife · doi:10.7554/elife.39404 · Cited by 79
Suboptimal choice in rats: Incentive salience attribution promotes maladaptive decision-making
Chow, J. J., Smith, A. P., Wilson, A. G., Zentall, T. R., & Beckmann, J. S. (2016) · Behavioural Brain Research · doi:10.1016/j.bbr.2016.12.013 · Cited by 71
Orbitofrontal neurons signal reward predictions, not reward prediction errors
Stalnaker, T. A., Liu, T., Takahashi, Y. K., & Schoenbaum, G. M. (2018) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2018.01.013 · Cited by 60
The Computational Anatomy of Visual Neglect
Parr, T., & Friston, K. J. (2017) · Cerebral Cortex · doi:10.1093/cercor/bhx316 · Cited by 57
Self-citationWhat does the free energy principle tell us about the brain?Preprint
Gershman, S. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.07945 · Cited by 46
Whisking mechanics and active sensing
Bush, N. E., Solla, S. A., & Hartmann, M. J. Z. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.08.001 · Cited by 43
Luciw, M., Kompella, V. R., Kazerounian, S., & Schmidhuber, J. (2013) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2013.00009 · Cited by 38
Associative Reinforcement Learning: A Generate and Test Algorithm
Kaelbling, L. P. (1994) · Machine Learning · doi:10.1023/a:1022642026684 · Cited by 29
Gambling in rhesus macaques (Macaca mulatta): The effect of cues signaling risky choice outcomes
Smith, T. R., Beran, M. J., & Young, M. E. (2017) · Learning & Behavior · doi:10.3758/s13420-017-0270-5 · Cited by 21
Intrinsically Motivated Exploration of Learned Goal Spaces
Laversanne-Finot, A., Péré, A., & Oudeyer, P. (2021) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2020.555271 · Cited by 17
Cited by
In vitro neurons learn and exhibit sentience when embodied in a simulated game-world
Kagan, B. J., Kitchen, A. C., Tran, N. T., Habibollahi, F., Khajehnejad, M., Parker, B. J., Bhat, A., Rollo, B., Razi, A., & Friston, K. J. (2022) · Neuron · doi:10.1016/j.neuron.2022.09.001 · Cited by 400
Self-citationA step-by-step tutorial on active inference and its application to empirical data
Smith, R., Friston, K. J., & Whyte, C. J. (2022) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2021.102632 · Cited by 246
Self-citationCorcoran, A. W., Pezzulo, G., & Hohwy, J. (2020) · Biology & Philosophy · doi:10.1007/s10539-020-09746-2 · Cited by 161
(2021) · UCL Discovery (University College London) · Cited by 141
Learning action-oriented models through active inference
Tschantz, A., Seth, A. K., & Buckley, C. L. (2020) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1007805 · Cited by 121
Metadata
- Authors
- Schwartenbeck, P., Passecker, J., Hauser, T. U., FitzGerald, T. H., Kronbichler, M., & Friston, K. J.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2019-05-10
- Volume
- 8
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W2892266804
- Cited by
- 254
- References
- 98
- FWCI
- 14.04
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.