An Active Inference Approach to Modeling Structure Learning: Concept Learning as an Example Case
Open accessThe DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Abstract
Within computational neuroscience, the algorithmic and neural basis of structure learning remains poorly understood. Concept learning is one primary example, which requires both a type of internal model expansion process (adding novel hidden states that explain new observations), and a model reduction process (merging different states into one underlying cause and thus reducing model complexity via meta-learning). Although various algorithmic models of concept learning have been proposed within machine learning and cognitive science, many are limited to various degrees by an inability to generalize, the need for very large amounts of training data, and/or insufficiently established biological plausibility. Using concept learning as an example case, we introduce a novel approach for modeling structure learning-and specifically state-space expansion and reduction-within the active inference framework and its accompanying neural process theory. Our aim is to demonstrate its potential to facilitate a novel line of active inference research in this area. The approach we lay out is based on the idea that a generative model can be equipped with extra (hidden state or cause) "slots" that can be engaged when an agent learns about novel concepts. This can be combined with a Bayesian model reduction process, in which any concept learning-associated with these slots-can be reset in favor of a simpler model with higher model evidence. We use simulations to illustrate this model's ability to add new concepts to its state space (with relatively few observations) and increase the granularity of the concepts it currently possesses. We also simulate the predicted neural basis of these processes. We further show that it can accomplish a simple form of "one-shot" generalization to new stimuli. Although deliberately simple, these simulation results highlight ways in which active inference could offer useful resources in developing neurocomputational models of structure learning. They provide a template for how future active inference research could apply this approach to real-world structure learning problems and assess the added utility it may offer.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“However, a number of approaches require much larger amounts of data than humans do to learn new concepts (Geman et al., 1992; Lecun et al., 1998; LeCun et al., 2015; Hinton et al., 2012; Mnih et al., 2015).”
BackgroundThe free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120
Self-citationRecognition-by-components: A theory of human image understanding.
Biederman, I. (1987) · Psychological Review · doi:10.1037/0033-295x.94.2.115 · Cited by 5,621
Gigerenzer, G., & Gaissmaier, W. (2011) · Annual Review of Psychology · doi:10.1146/annurev-psych-120709-145346 · Cited by 4,037
Neural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578 · cited 2×
“However, a number of approaches require much larger amounts of data than humans do to learn new concepts (Geman et al., 1992; Lecun et al., 1998; LeCun et al., 2015; Hinton et al., 2012; Mnih et al., 2015).”
BackgroundHuman-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
Tononi, G., & Cirelli, C. (2014) · Neuron · doi:10.1016/j.neuron.2013.12.025 · Cited by 2,356 · cited 2×
“This is consistent with work suggesting that, during sleep, many (but not all) synaptic strength increases acquired in the previous day are attenuated (Tononi & Cirelli, 2014).”
BackgroundResultProbabilistic machine learning and artificial intelligence
Ghahramani, Z. (2015) · Nature · doi:10.1038/nature14541 · Cited by 1,995 · cited 2×
“Other recent hierarchical Bayesian approaches in cognitive science have made progress in this domain, however, by modeling concepts as types of probabilistic programs (Ghahramani, 2015; Goodman, Tenenbaum, & Gerstenberg, 2015; Lake et al., 2015).”
BackgroundBarsalou, L. W. (1983) · Memory & Cognition · doi:10.3758/bf03196968 · Cited by 1,879 · cited 2×
“…Mnih et al., 2015), and do not learn the rich structure that allows humans to use concept knowledge in a wide variety of generalizable functions (Barsalou, 1983; Biederman, 1987; Feldman, 1997; Jern & Kemp, 2013; A. B. Markman & Makin, 1998; Osherson & Smith, 1981; Ward, 1994; Williams &…”
BackgroundCompetitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471 · cited 2×
“For example, the adaptive resonance theory (ART) model (Grossberg, 1987) was designed to incorporate top-down attentional mechanisms and feedback mechanisms to address a fundamental knowledge acquisition problem—the temporal instability of previously learned information that can occur when a system also remains suffic…”
MethodHippocampal place-cell sequences depict future paths to remembered goals
Pfeiffer, B. E., & Foster, D. J. (2013) · Nature · doi:10.1038/nature12112 · Cited by 1,216
Sharot, T. (2011) · Current Biology · doi:10.1016/j.cub.2011.10.030 · Cited by 1,187
The importance of shape in early lexical learning
Landau, B., Smith, L. B., & Jones, S. S. (1988) · Cognitive Development · doi:10.1016/0885-2014(88)90014-7 · Cited by 1,174
Active inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156
Self-citationThe adaptive nature of human categorization.
Anderson, J. R. (1991) · Psychological Review · doi:10.1037/0033-295x.98.3.409 · Cited by 1,069
The hippocampus as a predictive map
Stachenfeld, K. L., Botvinick, M. M., & Gershman, S. J. (2017) · Nature Neuroscience · doi:10.1038/nn.4650 · Cited by 1,000
Variational free energy and the Laplace approximation
Friston, K., Mattout, J., Trujillo-Barreto, N., Ashburner, J., & Penny, W. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2006.08.035 · Cited by 880
Self-citationWhat is intrinsic motivation? A typology of computational approaches
Oudeyer, P.-Y. (2007) · Frontiers in Neurorobotics · doi:10.3389/neuro.12.006.2007 · Cited by 811
Sleep, learning, and dreams: off-line memory reprocessing
Stickgold, R., Hobson, J. A., Fosse, R., & Fosse, M. (2001) · Science · doi:10.1126/science.1063530 · Cited by 762 · cited 1×
“The role of sleep and daydreaming in keeping “reserve” representational resources available for model expansion could therefore be especially important to concept learning—consistent with the known role of sleep in learning and memory (Stickgold et al., 2001; Walker and Stickgold, 2010; Perogamvros and Schwartz, 2012;…”
BackgroundFormal theory of creativity, fun, and intrinsic motivation (1990–2010)
Schmidhuber, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2056368 · Cited by 747
Every Good Regulator of a System Must Be a Model of That System
Conant, R. C., & Ashby, W. R. (1991) · Facets of Systems Science · doi:10.1007/978-1-4899-0718-9_37 · Cited by 712
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., ODoherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674
Self-citationPrefrontal cortex as a meta-reinforcement learning system
Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., & Botvinick, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0147-8 · Cited by 596 · cited 1×
“…abstract, task-general knowledge that results from training across different tasks, as recently discussed in the context of meta-reinforcement learning (Ritter, Wang, Kurth-Nelson, & Botvinick, 2018; J X Wang et al., 2016) with a putative link to the prefrontal cortex (Jane X. Wang et al., 2018).”
BackgroundLearning to reinforcement learn
Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M. (2016) · arXiv · doi:10.48550/arxiv.1611.05763 · Cited by 579
McKay, R. T., & Dennett, D. C. (2009) · Behavioral and Brain Sciences · doi:10.1017/s0140525x09990975 · Cited by 474
A rational analysis of rule-based concept learning
Goodman, N. D., Tenenbaum, J. B., Feldman, J., & Griffiths, T. L. (2008) · Cognitive Science · doi:10.1080/03640210701802071 · Cited by 469
The graphical brain: Belief propagation and active inference
Friston, K. J., Parr, T., & de Vries, B. (2017) · Network Neuroscience · doi:10.1162/netn_a_00018 · Cited by 415
Self-citationBarto, A., Mirolli, M., & Baldassarre, G. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00907 · Cited by 397 · cited 1×
“Yet, while infants may initially be somewhat restricted to passively observing their environment, observations quickly come under the control of action (perhaps bootstrapping in a useful way on passive observations to first acquire sensorimotor contingencies; Deci and Ryan, 1985; Oudeyer, 2007; Schmidhuber, 2010; Bart…”
BackgroundRational approximations to rational models: alternative algorithms for category learning
Sanborn, A. N., Griffiths, T. L., & Navarro, D. J. (2010) · Psychological Review · doi:10.1037/a0020511 · Cited by 369 · cited 1×
“For example, two previously proposed models – the “rational model” (Anderson 1991; Sanborn et al. 2010) and the SUSTAIN model (Love et al. 2004) – both describe concept acquisition as involving cluster creation mechanisms that depend on statistical regularities during learning and that use…”
MethodLearning latent structure: carving nature at its joints
Gershman, S. J., & Niv, Y. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.02.008 · Cited by 326
Waking and dreaming consciousness: neurobiological and functional considerations.
Hobson, J. A., & Friston, K. J. (2012) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2012.05.003 · Cited by 287
Self-citationThe roles of the reward system in sleep and dreaming
Perogamvros, L., & Schwartz, S. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.05.010 · Cited by 239
Overnight alchemy: sleep-dependent memory evolution
Walker, M. P., & Stickgold, R. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2762-c1 · Cited by 229
The dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219
Self-citationHow memory replay in sleep boosts creative problem-solving
Lewis, P. A., Knoblich, G., & Poe, G. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.03.009 · Cited by 210 · cited 1×
“, as appears to happen in the 1174 phenomenon of hippocampal replay; (Pfeiffer and Foster, 2013; Feld and Born, 1175 2017; Lewis et al., 2018)); this type of learning can accomplish a model reduction 1176 process in which redundant model parameters are identified and removed to 1177 prevent model over-fitting and prom…”
BackgroundHow language shapes the cultural inheritance of categories
Gelman, S. A., & Roberts, S. O. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1621073114 · Cited by 200
Virtual reality and consciousness inference in dreaming
Hobson, J. A., Hong, C. C.-H., & Friston, K. J. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01133 · Cited by 197 · cited 3×
“For a technical description of Bayesian model reduction techniques and their proposed neural implementation, see (Hobson and Friston, 2012; Hobson et al., 2014; Friston et al., 2017b); see the left panel of Figure 2 for additional details).”
BackgroundMethodInfluentialSelf-citationThe computational nature of memory modification
Gershman, S. J., Monfils, M.-H., Norman, K. A., & Niv, Y. (2017) · eLife · doi:10.7554/elife.23763 · Cited by 172
Scene Construction, Visual Foraging, and Active Inference
Mirza, M. B., Adams, R. A., Mathys, C. D., & Friston, K. J. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00056 · Cited by 165
Self-citationGradual extinction prevents the return of fear: implications for the discovery of state
Gershman, S. J., Jones, C. E., Norman, K. A., Monfils, M.-H., & Niv, Y. (2013) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2013.00164 · Cited by 160
GABA depolarization is required for experience-dependent synapse unsilencing in adult-born neurons
Chancey, J. H., Adlaf, E. W., Sapp, M. C., Pugh, P. C., Wadiche, J. I., & Overstreet-Wadiche, L. S. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.0781-13.2013 · Cited by 152
Neuronal message passing using mean-field, bethe, and marginal approximations
Parr, T., Markovic, D., Kiebel, S. J., & Friston, K. J. (2019) · Scientific Reports · doi:10.1038/s41598-018-38246-3 · Cited by 128
Self-citationLearning action-oriented models through active inference
Tschantz, A., Seth, A. K., & Buckley, C. L. (2020) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1007805 · Cited by 121
Toward the neural implementation of structure learning
Tervo, D. G. R., Tenenbaum, J. B., & Gershman, S. J. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.01.014 · Cited by 106
Smith, R., Lane, R. D., Parr, T., & Friston, K. J. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.09.002 · Cited by 89
Self-citationGeneralisation of structural knowledge in the hippocampal-entorhinal system
Whittington, J. C. R., Muller, T. H., Mark, S., Barry, C., & Behrens, T. E. J. (2018) · arXiv · doi:10.48550/arxiv.1805.09042 · Cited by 36
Cited by
A step-by-step tutorial on active inference and its application to empirical data
Smith, R., Friston, K. J., & Whyte, C. J. (2022) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2021.102632 · Cited by 246
Self-citationThe empirical status of predictive coding and active inference
Hodson, R., Mehta, M., & Smith, R. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105473 · Cited by 69
Self-citationArnaldo, I., Corcoran, A. W., Friston, K. J., & Ramstead, M. J. D. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104590 · Cited by 56
Self-citationNeural dynamics under active inference: plausibility and efficiency of information processing
Da Costa, L., Parr, T., Sengupta, B., & Friston, K. (2021) · Entropy · doi:10.3390/e23040454 · Cited by 38
Self-citationReward maximization through discrete active inference
Da Costa, L., Sajid, N., Parr, T., Friston, K., & Smith, R. (2023) · Neural Computation · doi:10.1162/neco_a_01574 · Cited by 35
Self-citation
Metadata
- Authors
- Smith, R., Schwartenbeck, P., Parr, T., & Friston, K. J.
- Venue
- Frontiers in Computational Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2020-05-19
- Volume
- 14
- Pages
- 41-41
- ISSN
- 1662-5188 (ISSN-L)
- OpenAlex
- W3026687585
- Cited by
- 99
- References
- 47
- FWCI
- 6.63
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.