The DOI matches its Crossref record; 153 of the 153 references it cites were found in a registry or an index.
The why, what, where, when and how of goal-directed choice: neuronal and computational principles
Open accessAbstract
The central problems that goal-directed animals must solve are: 'What do I need and Why, Where and When can this be obtained, and How do I get it?' or the H4W problem. Here, we elucidate the principles underlying the neuronal solutions to H4W using a combination of neurobiological and neurorobotic approaches. First, we analyse H4W from a system-level perspective by mapping its objectives onto the Distributed Adaptive Control embodied cognitive architecture which sees the generation of adaptive action in the real world as the primary task of the brain rather than optimally solving abstract problems. We next map this functional decomposition to the architecture of the rodent brain to test its consistency. Following this approach, we propose that the mammalian brain solves the H4W problem on the basis of multiple kinds of outcome predictions, integrating central representations of needs and drives (e.g. hypothalamus), valence (e.g. amygdala), world, self and task state spaces (e.g. neocortex, hippocampus and prefrontal cortex, respectively) combined with multi-modal selection (e.g. basal ganglia). In our analysis, goal-directed behaviour results from a well-structured architecture in which goals are bootstrapped on the basis of predefined needs, valence and multiple learning, memory and planning mechanisms rather than being generated by a singular computation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1. The DAC theory of mind and brain (see [10] for a review). Left: highly abstract representation of the DAC architecture. DAC proposes that the brain is organized as a three-layered control structure with tight coupling within and between these layers distinguishing: the soma (SL) and the reactive (RL), adaptive (AL) and contextual (CL) layers. Across these layers, a columnar organization exists that deals with the processing of states of the World or exteroception (left, red), the self or interoception (middle, blue) and action (right, green). See text for further explanation. The reactive layer: the RL comprises dedicated behaviour systems (BS) that combine predefined sensorimotor mappings with drive reduction mechanisms that are predicated on the needs of the body (SL). Right lower panel: each BS follows homeostatic principles supporting the self-essential functions (SEF) of the body (SL). In order to map needs into behaviours, the strength of the essential variables served by the BSs, SEFs, have a specific distribution in task-space called an ‘affordance gradient’. In this example, we consider the (internally represented) ‘attractive force’ of the home position supporting the security SEF or of open space defining the exploration SEF. The values of the respective SEFs are defined by the difference between the sensed value of the affordance gradient (red) and its desired value given the prevailing needs (blue). The regulator of each BS defines the next action as to perform a gradient ascent on the SEF. An integration and action selection process across the different BSs forces a strict winner-take-all decision that defines the specific behaviour emitted. The allostatic controller of the RL regulates the internal homeostatic dynamic of the BSs to set priorities defined by needs and environmental opportunities through the modulation of the affordance gradients, desired values of SEFs and/or the integration process. The adaptive layer: the AL acquires a state space of the agent–environment interaction and shapes action. The learning dynamic of AL is constrained by the SEFs of the RL that define value. The AL crucially contributes to exosensing by allowing the processing of states of distal sensors, e.g. vision and audition, which are not predefined but rather are tuned in somatic time to properties of the interaction with the environment. Acquired sensor and motor states are in turn associated through the valence states signalled by the RL. The contextual layer: the core processes of the CL are divided between a task-model and a self-model. The CL expands the time horizon in which the agent can operate through the use of sequential short-term and long-term memory (STM and LTM) systems respectively. These memory systems operate on integrated sensorimotor representations that are generated by the AL and acquire, retain and express goal-oriented action regulated by the RL. The CL comprises a number of processes (right upper panel): (a) when the discrepancy between predicted and encountered sensory states falls below a STM acquisition threshold, the perceptual predictions (red circle) and motor activity (green rectangle) generated by AL are stored in STM as a, so-called, segment. The STM acquisition threshold is defined by the time-averaged reconstruction error of the perceptual learning system of AL. (b) If a goal state (blue flag) is reached, e.g. reward or punishment, the content of STM is retained in LTM as a sequence conserving its order, goal state and valence marker, e.g. aversive or appetitive, and STM is reset. Every sequence is thus labelled with respect to the specific goal it pertains to and its valence marker. (c) If the outputs generated by the RL and AL to action selection are sub-threshold, the AL perceptual predictions are matched against those stored in LTM. (d) The CL selected action is defined as a weighted sum over the segments of LTM. (e) The contribution of LTM segments to decision-making depends on four factors: perceptual evidence, memory chaining, the distance to the goal state and valence. Working memory (WM) of the CL is defined by the memory dynamics that represents these factors. Active segments that contributed to the selected action are associated with those that were previously active establishing rules for future chaining. The self-model component of the CL monitors task performance and develops (re)descriptions of task dynamics anchored in the self. In this way, the system generates meta-representational knowledge that forms autobiographical memory. This aspect of the DAC CL is not further considered in this paper.](/paper-figures/21d6bfd3592c4a3dbfffc67c1ba6b12f/the-why-what-where-when-and-how-of-goal-directed-choice-neuronal-and-figure-1.jpg)
![Figure 2. Proposed brain architecture representing the neuronal substrate of goal-directed behaviour and its relation to the neurorobotic DAC architecture. The hippocampal formation is proposed to code the organism's world state space (red), the prefrontal cortex (especially its medial and orbitofrontal aspects) to represent task space (i.e. rules, constraints, goals and values of cues and action options, purple) and the striatum (and downstream structures of the basal ganglia) to mediate action selection. In this scheme, the hypothalamus and brain stem contain sensor systems monitoring homeostatic variables and providing information about the motivational needs of the organism that define the pursuit of needs and goals (blue). The arrow from striatum to thalamus represents an indirect projection. The hypothalamic efferents are modelled after those traced for the lateral hypothalamus in relation to feeding behaviour and do not apply to hypothalamic areas in general. For the sake of clarity, the scheme's anatomic connections are by no means complete. For instance, outputs from prefrontal cortex and basal ganglia to the brain stem, or several afferent inputs to amygdala and VTA, have not been included, while several hypothalamic nuclei project directly to ventromedial prefrontal areas. Sensory inputs reach the hippocampus via intermediate stations (parahippocampal areas; not shown) and are supplemented with frontal cortical inputs converging on these intermediate areas. Furthermore, the motor cortices are meant to include premotor, supplementary motor and frontal oculomotor areas (based on [40–42]).](/paper-figures/71bcd641609344238bd6e4f3088881f2/the-why-what-where-when-and-how-of-goal-directed-choice-neuronal-and-figure-2.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In most psychological theories, drives are usually linked to basic motivations and homoeostasis (e.g., interoceptive hunger or thirst), while goals are considered to have more elaborate (exteroceptive) sensory and cognitive aspects (e.g., dining in a restaurant), but their relations are often unclear from a mechanistic viewpoint (Pezzulo et al., 2014b, Verschure et al., 2014).”
From Active Inference, homeostatic regulation and adaptive behavioural control (2015) · cites this paper as Verschure et al., 2014
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 1×
“CSL is consistent with ‘predictive brain’ frameworks and ‘free energy minimization’ principles [29,30].”
MethodReal-Time Obstacle Avoidance for Manipulators and Mobile Robots
Khatib, O. (1986) · The International Journal of Robotics Research · doi:10.1177/027836498600500106 · Cited by 7,894 · cited 1×
“This idea has its roots in Gestalt psychology [20] and is a standard approach towards robot navigation [21] but generalizes towards optimal path planning in mobile robots [22] and social interaction in humanoid robots [23].”
BackgroundSearle, J. R. (1980) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00005756 · Cited by 6,954 · cited 1×
“The AL allows the agent to overcome the predefined behavioural repertoire of the RL and to engage an a priori unpredictable world, solving the notorious symbol grounding that led to the demise of classical AI [3,28].”
BackgroundCognitive maps in rats and men
Tolman, E. C. (1948) · Psychological Review · doi:10.1037/h0061626 · Cited by 6,641 · cited 1×
“The first view holds that the hippocampus codes an agent’s position in space, as inspired by the Cognitive Map theory of Tolman [63] as well as the body of place-cell research initiated by O’Keefe & Dostrovsky [64].”
MethodThe hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat
O'Keefe, J., & Dostrovsky, J. (1971) · Brain Research · doi:10.1016/0006-8993(71)90358-1 · Cited by 6,151
Episodic memory: from mind to brain
Tulving, E. (2002) · Annual Review of Psychology · doi:10.1146/annurev.psych.53.100901.135114 · Cited by 4,950 · cited 1×
“The second notion accounts for hippocampal function as a ‘recorder’ of experience—an organ for the formation of episodic memories that would be transferred, with the passage of time, to neocortical sites where also generalization (semanticization) of memory may take place [65].”
BackgroundPredictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564 · cited 1×
“Building on previous models that framed the basal ganglia as an actor–critic architecture for (model-free) reinforcement learning [120,121], we recently argued that especially the ‘Critic’ (goal-predictive) function of the striatum is well supported by the data, whereas its implementation of the ‘Actor’ component is m…”
ResultAffective Neuroscience: The Foundations of Human and Animal Emotions
Panksepp, J. (1998) · doi:10.1093/oso/9780195096736.001.0001 · Cited by 4,428
(1991) · Choice Reviews Online · doi:10.5860/choice.28-5376 · Cited by 4,218
(1998) · Cited by 4,012
Research works printed but not linked: 8
- Verschure PFMJ, 2013, Formal minds and biological brains II: from the mirage of intelligence to a science and engineering of consciousness, IEEE Expert., 28, 33
- Panksepp J, 2012, The archaeology of mind: neuroevolutionary origins of human emotions (Norton series on interpersonal neurobiology)
- Sanchez-Fibla MS, 2010, IEEE/RSJ Int. Conf. 18–22 October 2010
- Miller S, 1928, Sur une forme particuliere des reflexes conditionnels, C.R. Seances Soc. Polonaise Biol., 49, 1155
- Ringwald M, 2007, ECMR 2007–3rd European Conf. on Mobile Robots
- Simerly R, 1995, The rat nervous system, 353
- Gray JA, 2000, The neuropsychology of anxiety: an enquiry into thé functions of thesepto-hippocampal system
- Verschure PFMJ, 2012, The distributed adaptive control architecture of the mind, brain, body nexus, Biol. Inspired Cogn. Archit., 1, 55
Cited by
Sorted by citations
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., O⿿Doherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674
Active Inference, homeostatic regulation and adaptive behavioural control
Pezzulo, G., Rigoli, F., & Friston, K. (2015) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2015.09.001 · Cited by 650
“Some of these architectures also include a (simplified) internal physiology that grounds higher-level goals (Verschure et al., 2014).”
InfluentialCaligiore, D., Pezzulo, G., Baldassarre, G., Bostan, A. C., Strick, P. L., Doya, K., Helmich, R. C. G., Dirkx, M. F., Houk, J. C., Jörntell, H., et al. (2016) · The Cerebellum · doi:10.1007/s12311-016-0763-3 · Cited by 445
Navigating the Affordance Landscape: Feedback Control as a Process Model of Behavior and Cognition
Pezzulo, G., & Cisek, P. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.03.013 · Cited by 419
Hierarchical active inference: A theory of motivated control
Pezzulo, G., Rigoli, F., & Friston, K. J. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.01.009 · Cited by 334
“Calling on early cybernetic models [16–18], the view that the brain uses control hierarchies has inspired many recent proposals [19–25]; for example, hierarchical temporal structures [26], hierarchical reinforcement learning [9], hierarchical mixture of experts [23], distributed adaptive control [8,27] and hierarchica…”
Self-citationLevin, M. G., Pezzulo, G., & Finkelstein, J. M. (2017) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-071114-040647 · Cited by 292
“, from biophysically realistic neural networks to more abstract models; from single-neuron models to large neural networks) have often been useful, as they can provide detailed process models of neural computations and produce quantitative and empirically testable predictions (186).”
Pezzulo, G., & Levin, M. G. (2016) · Journal of The Royal Society Interface · doi:10.1098/rsif.2016.0555 · Cited by 245
Embodied choice: how action influences perceptual decision making
Lepora, N. F., & Pezzulo, G. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004110 · Cited by 209
“We make the case that as cognitive processes are studied in ever more ecological contexts, the importance of the cognitive embodiment will become increasingly apparent [65–67].”
BackgroundSelf-citationShin, H., Son, Y., Chae, U., Kim, J., Choi, N., Lee, H. J., Woo, J., Cho, Y., Yang, S. H., Lee, C. J., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-11628-5 · Cited by 174
Pezzulo, G., & Levin, M. G. (2015) · Integrative Biology · doi:10.1039/c5ib00221d · Cited by 170
Decision-Making From the Animal Perspective: Bridging Ecology and Subjective Cognition
Budaev, S. V., Jørgensen, C., Mangel, M., Eliassen, S., & Giske, J. (2019) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2019.00164 · Cited by 130
The evolution of brain architectures for predictive coding and active inference
Pezzulo, G., Parr, T., & Friston, K. J. (2021) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2020.0531 · Cited by 136
“Rather, older solutions largely remain available and can be selected in the appropriate conditions—thus rendering action selection context-sensitive [5,63].”
BackgroundCorcoran, A. W., Pezzulo, G., & Hohwy, J. (2020) · Biology & Philosophy · doi:10.1007/s10539-020-09746-2 · Cited by 161
Agnès Lacreuse, A., Raz, N., Schmidtke, D., Hopkins, W. D., & Herndon, J. G. (2020) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2019.0618 · Cited by 105
“Living organisms adapt to their environment through processing information [1,2].”
BackgroundBiomaterials via Peptide Assembly: Design, Characterization, and Application in Tissue Engineering
Gray, V. P., Amelung, C. D., Duti, I. J., Laudermilch, E. G., Letteri, R. A., & Lampe, K. J. (2021) · Acta Biomaterialia · doi:10.1016/j.actbio.2021.10.030 · Cited by 92
Active inference and robot control: a case study
Pio‐Lopez, L., Nizard, A., Friston, K. J., & Pezzulo, G. (2016) · Journal of The Royal Society Interface · doi:10.1098/rsif.2016.0616 · Cited by 91
How deep is the brain? The shallow brain hypothesis
Suzuki, M., Pennartz, C. M. A., & Aru, J. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00756-z · Cited by 79
Self-citationEvaluating the Brain Disease Model of Addiction
Heather, N., Field, M., Moss, A. C., & Satel, S. L. (2022) · Routledge eBooks · doi:10.4324/9781003032762 · Cited by 86
Moulin-Frier, C., Fischer, T., Petit, M., Pointeau, G., Puigbò, J.-Y., Pattacini, U., Low, S. C., Camilleri, D., Nguyen, P. D. H., Hoffmann, M., et al. (2017) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2017.2754143 · Cited by 82
“In this paper, we adopt the principles of the Distributed Adaptive Control theory of the mind and the brain (DAC, [39], [40]).”
Developmental perspectives on risky and impulsive choice
Rosenbaum, G. M., & Hartley, C. A. (2018) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2018.0133 · Cited by 75
Rusu, S. I., & Pennartz, C. M. A. (2019) · Hippocampus · doi:10.1002/hipo.23167 · Cited by 63
“A key proposition holds that the HPC does not encode a “task space” as proposed for PFC, but a representation of world states (including the subject's body state; cf. Verschure et al., 2014).”
BackgroundGoal-directed actions transiently depend on dorsal hippocampus
Bradfield, L. A., Leung, B. K., Boldt, S., Liang, S., & Balleine, B. W. (2020) · Nature Neuroscience · doi:10.1038/s41593-020-0693-8 · Cited by 65
Maffei, G., Santos-Pata, D., Marcos, E., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.10.004 · Cited by 60
Donnarumma, F., Maisto, D., & Pezzulo, G. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004864 · Cited by 62
Divide et impera : subgoaling reduces the complexity of probabilistic inference and problem solving
Maisto, D., Donnarumma, F., & Pezzulo, G. (2015) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.1335 · Cited by 59
Functional Oxides for Photoneuromorphic Engineering: Toward a Solar Brain
Pérez‐Tomás, A. (2019) · Advanced Materials Interfaces · doi:10.1002/admi.201900471 · Cited by 51
Treating Brain Disorders by Targeting Adult Neural Stem Cells
Bao, H., & Song, J. (2018) · Trends in Molecular Medicine · doi:10.1016/j.molmed.2018.10.001 · Cited by 53
Synthetic consciousness: the distributed adaptive control perspective
Verschure, P. F. M. J. (2016) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2015.0448 · Cited by 54
Model-Based Approaches to Active Perception and Control
Pezzulo, G., Donnarumma, F., Iodice, P., Maisto, D., & Stoianov, I. P. (2017) · Entropy · doi:10.3390/e19060266 · Cited by 42
A thalamocortical circuit for updating action-outcome associations
Fresno, V., Parkes, S. L., Faugère, A., Coutureau, É., & Wolff, M. (2019) · eLife · doi:10.7554/elife.46187 · Cited by 42
“However, mounting experimental evidence indicates that functional interactions between prefrontal and subcortical areas are essential to support these functions (Balleine and O’Doherty, 2010; Verschure et al., 2014).”
BackgroundZhang, S., Wang, W., Zhornitsky, S., & Li, C. R. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00344 · Cited by 41
Lateral parietal cortex in the generation of behavior: Implications for apathy
Tumati, S., Martens, S., de Jong, B. M., & Aleman, A. (2018) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2018.12.003 · Cited by 40
“the goal and the reason for its choice; the ‘what’ component represents the broad plan of action; the ‘when’ and ‘where’ components reflect the spatiotemporal aspects of the actions; the ‘how’ component describes the detailed motor sequences to be performed (Verschure et al., 2014).”
Prefrontal Goal Codes Emerge as Latent States in Probabilistic Value Learning
Stoianov, I., Genovesio, A., & Pezzulo, G. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00886 · Cited by 44
“…Solway and Botvinick, 2012; Pezzulo et al., 2013) and active inference (Friston, 2010; Clark, 2013), where they have a key role in guiding action selection and control (Lepora and Pezzulo, 2015, Pezzulo and Castelfranchi, 2009, Pezzulo et al., 2014a, Pezzulo et al., 2014b, Verschure et al., 2014).”
BackgroundA Liaison Brought to Light: Cerebellum-Hippocampus, Partners for Spatial Cognition
Rondi‐Reig, L., Paradis, A., & Fallahnezhad, M. (2022) · The Cerebellum · doi:10.1007/s12311-022-01422-3 · Cited by 39
“However, the fact that these were not correlated with possible motor deficits confirmed the cerebellum as a crucial brain region for goal-directed navigation, independently of its motor role ([21–23] but also see [24]).”
BackgroundThe Synthetic Psychology of the Self
Prescott, T., & Camilleri, D. (2018) · Intelligent Systems, Control and Automation: Science and Engineering · doi:10.1007/978-3-319-97550-4_7 · Cited by 29
“For the past seven years, together with European colleagues, we have also been incorporating models of key brain systems into a brain-inspired control architecture for the iCub robot (Figure 1) called distributed adaptive control (DAC), developed by Paul Verschure and colleagues [69-71].”
BackgroundModel-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysis
Stoianov, I. P., Pennartz, C. M. A., Lansink, C. S., & Pezzulo, G. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006316 · Cited by 38
“This reliance on a relatively low-dimensional state space makes spatial navigation quite special and tractable, both in machine learning and the brain; it is thus possible that brain areas that support goal-directed processing outside navigation domains rely on more complex computational solutions that remain to be fu…”
BackgroundReadiness to remember: predicting variability in episodic memory
Madore, K. P., & Wagner, A. D. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.05.006 · Cited by 37
Renewal of goal direction with a context change after habit learning.
Steinfeld, M. R., & Bouton, M. E. (2021) · Behavioral Neuroscience · doi:10.1037/bne0000422 · Cited by 34
The principles of goal-directed decision-making: from neural mechanisms to computation and robotics
Pezzulo, G., Verschure, P. F. M. J., Balkenius, C., & Pennartz, C. M. A. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0470 · Cited by 35
Memory and mental time travel in humans and social robots
Prescott, T. J., Camilleri, D., Martínez-Hernández, U., Damianou, A., & Lawrence, N. D. (2019) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2018.0025 · Cited by 37
“This is already happening in the DAC architecture where action plans, represented as graphical models, can be accessed based on goal cues extracted from human speech.”
MethodInfluentialLearning and Judgment Can Be Affected by Predisposed Fearfulness in Laying Hens
Haas, E. N. D., Lee, C., & Rodenburg, T. B. (2017) · Frontiers in Veterinary Science · doi:10.3389/fvets.2017.00113 · Cited by 32
The ventral hippocampus is involved in multi‐goal obstacle‐rich spatial navigation
Contreras, M., Pelc, T., Llofriu, M., Weitzenfeld, A., & Fellous, J. (2018) · Hippocampus · doi:10.1002/hipo.22993 · Cited by 37
Socially adaptive cognitive architecture for human-robot collaboration in industrial settings
Freire, I. T., Guerrero-Rosado, O., Amil, A. F., & Verschure, P. F. M. J. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1248646 · Cited by 34
Active inference and cognitive-emotional interactions in the brain
Pezzulo, G., Barca, L., & Friston, K. J. (2015) · Behavioral and Brain Sciences · doi:10.1017/s0140525x14001009 · Cited by 35
Hypothalamic Responses to Cocaine and Food Cues in Individuals with Cocaine Dependence
Zhang, S., Zhornitsky, S., Le, T. M., & Li, C.-S. R. (2019) · The International Journal of Neuropsychopharmacology · doi:10.1093/ijnp/pyz044 · Cited by 30
“The hypothalamus receives heavy dopaminergic projections from the midbrain and represents a key brain region in regulating arousal, food intake, and sexual drive and in supporting reward and affective responses during a wide range of goal-directed behaviors (DiLeone et al., 2003; Watabe-Uchida et al., 2012; Sternson,…”
BackgroundUnderstanding brain functional architecture through robotics
Prescott, T. J., & Wilson, S. P. (2023) · Science Robotics · doi:10.1126/scirobotics.adg6014 · Cited by 31
“Broadly speaking, these controllers are arranged with fast but inflexible controllers at the bottom and slow but flexible ones at the top (4, 7).”
BackgroundCarvalho, F. M., Chaim, K. T., Sanchez, T. A., & de Araújo, D. B. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00268 · Cited by 23
“Efficient interaction between an agent and its surrounding relies heavily on the use of prospective internal models to enhance processing of expected stimuli, and on the ability to update these models to achieve the desired outcome (Verschure et al., 2014).”
BackgroundMACHINE INTELLIGENCE. ESSAYS ON THE THEORY OF MACHINE LEARNING AND ARTIFICIAL INTELLIGENCE
Шумский, С., & Shumskiy, S. (2019) · Publishing Center RIOR eBooks · doi:10.29039/02011-1 · Cited by 22
Nonparametric Problem-Space Clustering: Learning Efficient Codes for Cognitive Control Tasks
Maisto, D., Donnarumma, F., & Pezzulo, G. (2016) · Entropy · doi:10.3390/e18020061 · Cited by 20
“An open problem in this field is how task-relevant representations might be created in the first place, with alternative theories that link to statistical learning in the PFC proper or assign a role for the hippocampus in rapidly creating these clusters (or schemas [54]) and transferring them to cortical structures [1…”
BackgroundThe Morphospace of ConsciousnessPreprint
Arsiwalla, X. D., Moulin-Frier, C., Herreros, I., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2017) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/emhnp · Cited by 19
“In turn, evolution has shaped cognition in such a way so as to support the organism’s basic survival (using wakefulness/arousal) as well as higher-order drives (using awareness) associated to cooperation and competition in a multi-agent environment [80].”
BackgroundRaina, A., Puentes, L., Cagan, J., & McComb, C. (2021) · Journal of Mechanical Design · doi:10.1115/1.4051013 · Cited by 20
“Making decisions that optimize for specific objectives constitutes a critical skill in problem solving [11].”
BackgroundTowards a Synthetic Tutor Assistant: The EASEL Project and its Architecture
Vouloutsi, V., Blancas, M., Zucca, R., Omedas, P., Reidsma, D., Davison, D. P., Charisi, V., Wijnen, F., van der Meij, J., Evers, V., et al. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-42417-0_32 · Cited by 18
Embodied artificial intelligence through distributed adaptive control: An integrated framework
Moulin-Frier, C., Puigbò, J.-Y., Arsiwalla, X. D., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2017) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2017.8329825 · Cited by 26
“For this aim, we adopt the principles of the Distributed Adaptive Control (DAC) theory of the mind and brain [22], [23].”
MethodRomero‐Martínez, Á., Lila, M., Vitoria-Estruch, S., & Moya‐Albiol, L. (2018) · Journal of Interpersonal Violence · doi:10.1177/0886260518814263 · Cited by 18
How Pain Shapes Depression and Anxiety: A Hybrid Self-regulatory/Predictive Mind Perspective
Karoly, P. (2020) · Journal of Clinical Psychology in Medical Settings · doi:10.1007/s10880-019-09693-5 · Cited by 18
Learning and memory processes in behavioural addiction: A systematic review
Ngetich, R. K., Villalba-García, C., Soborun, Y., Vékony, T., Czakó, A., Demetrovics, Z., & Németh, D. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105747 · Cited by 22
de Maele, T. V., Dhoedt, B., Verbelen, T., & Pezzulo, G. (2024) · Nature Communications · doi:10.1038/s41467-024-54257-3 · Cited by 22
How? Why? What? Where? When? Who? Grounding Ontology in the Actions of a Situated Social Agent
Lallée, S., & Verschure, P. F. M. J. (2015) · Robotics · doi:10.3390/robotics4020169 · Cited by 16
Maier, M., Bañuelos, N. L., Ballester, B. R., Duarte, E., & Verschure, P. F. M. J. (2017) · International Conference on Rehabilitation Robotics · doi:10.1109/icorr.2017.8009371 · Cited by 17
“We adopted a finer gradient of impairment level as classically reported [31], [32].”
BackgroundMethodWerlen, E., & Jones, M. W. (2015) · Progress in brain research · doi:10.1016/bs.pbr.2015.03.002 · Cited by 16
Learning to select actions shapes recurrent dynamics in the corticostriatal system
Márton, C. D., Schultz, S. R., & Averbeck, B. B. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.09.008 · Cited by 18
Modeling the formation of social conventions from embodied real-time interactions
Freire, I. T., Moulin-Frier, C., Sánchez-Fibla, M., Arsiwalla, X. D., & Verschure, P. F. M. J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0234434 · Cited by 25
“As discussed in [26, 41], this will facilitate the integration of sensory-motor contingencies into a long-term memory that allows for learning of rules.”
Is Consciousness a Control Process?
Arsiwalla, X. D., Herreros, I., Moulin-Frier, C., Marti, S. B., & Verschure, P. F. M. J. (2016) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-61499-696-5-233 · Cited by 14
Towards Self-controlled Robots Through Distributed Adaptive Control
Puigbò, J.-Y., Moulin-Frier, C., & Verschure, P. F. M. J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-42417-0_52 · Cited by 13
Li, M., Fan, J., Cheng, S., Yang, L., Yang, L., Wang, L., Shang, Z., & Wan, H. (2021) · Animals · doi:10.3390/ani11072003 · Cited by 13
“, ‘What is the route from the current location to the known goal location?’) [2,3].”
BackgroundSyngap1 promotes cognitive function through regulation of cortical sensorimotor dynamics
Vaissière, T., Michaelson, S. D., Creson, T. K., Goins, J., Fürth, D., Balázsfi, D., Rojas, C., Golovin, R. M., Meletis, K., Miller, C. A., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-56125-0 · Cited by 13
“To explore this, we utilized behavioral paradigms that rely on passive (receptive) and active (generative) whisker sensing to drive perceptual learning 43,44 .”
Visual anticipation biases conscious decision making but not bottom-up visual processing
Mathews, Z., Cetnarski, R., & Verschure, P. F. M. J. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01443 · Cited by 14
Autonomous development of turn-taking behaviors in agent populations: A computational study
Moulin-Frier, C., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2015) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2015.7346139 · Cited by 11
“This rather strong hypothesis, which requires that each agent is able to identify a conspecific from the acoustic properties of its calls, is plausible for several social species (see e.g. [13] for the case of adult marmosets).”
BackgroundHatzipantelis, C. J., Langiu, M., Vandekolk, T. H., Pierce, T. L., Nithianantharajah, J., Stewart, G. D., & Langmead, C. J. (2020) · ACS Pharmacology & Translational Science · doi:10.1021/acsptsci.0c00117 · Cited by 13
A global framework for a systemic view of brain modeling
Alexandre, F. (2021) · Brain Informatics · doi:10.1186/s40708-021-00126-4 · Cited by 17
Maranesi, M., Bruni, S., Livi, A., Donnarumma, F., Pezzulo, G., & Bonini, L. (2019) · Scientific Reports · doi:10.1038/s41598-019-48216-y · Cited by 10
Goekoop, R., & de Kleijn, R. (2021) · Entropy · doi:10.3390/e23121701 · Cited by 9
“Another way to refer to such hierarchical predictive models of econiches is a ‘goal hierarchy’ [20,78].”
BackgroundRusu, S. I., Bos, J. J., Marchesi, P., Lankelma, J. V., Pica, I. F., Gentet, L. J., Joëls, M., & Pennartz, C. M. A. (2024) · iScience · doi:10.1016/j.isci.2024.109205 · Cited by 7
“32,100,101 How OFC replay may contribute to the formation of an internal model of task space 22,23,24 is difficult to tell at present.”
BackgroundRepresentation and the active consumer
Butlin, P. (2018) · Synthese · doi:10.1007/s11229-018-01941-9 · Cited by 9
Control systems theory revisited: new insights on the brain clocks of time-to-action
Ferber, S. G., Weller, A., & Soreq, H. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1171765 · Cited by 8
“While the elements within the parallel sequences are standardly ordered when operating within the same timeframe, that timeframe depends on the range of the required signal variability (Verschure et al., 2014; Sendhilnathan et al., 2021) for a given goal-oriented behavior.”
BackgroundThe Morphospace of Consciousness: Three Kinds of Complexity for Minds and Machines
Arsiwalla, X. D., Solé, R. V., Moulin-Frier, C., Herreros, I., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2023) · NeuroSci · doi:10.3390/neurosci4020009 · Cited by 10
“Social interactions play a crucial role in driving and regulating adaptive responses through behavioral feedback in both natural and artificial systems [20].”
BackgroundMoulin-Frier, C., Arsiwalla, X. D., Puigbò, J.-Y., Sánchez-Fibla, M., Duff, A., & Verschure, P. F. M. J. (2016) · neural information processing systems · Cited by 8
About the goal of a goals’ goal theory
Herreros, I., & Verschure, P. F. M. J. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1051952 · Cited by 8
Verbeke, W., & Masih, J. (2020) · Journal of Personal Selling & Sales Management · doi:10.1080/08853134.2019.1654389 · Cited by 6
“The task and action space is known as a set of rules, goals, and goal-predictive values (Verschure, Pennartz, and Pezzulo 2014) that are needed for using tools, solving concrete problems, and making choices.”
BackgroundMethodInfluentialOutline of a cybernetic theory of brain function based on neural timing nets
Cariani, P. (2015) · Kybernetes · doi:10.1108/k-11-2014-0242 · Cited by 7
Modeling the Formation of Social Conventions in Multi-Agent Populations.Preprint
Freire, I. T., Moulin-Frier, C., Sánchez-Fibla, M., Arsiwalla, X. D., & Verschure, P. F. M. J. (2018) · arXiv (Cornell University) · Cited by 7
Epistemic Autonomy: Self-supervised Learning in the Mammalian Hippocampus
Santos-Pata, D., Amil, A. F., Raikov, I., Rennó‐Costa, C., Mura, A., Soltész, I., & Verschure, P. F. M. J. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.03.016 · Cited by 7
“…network of the cortico-hippocampal system is a generic substrate for self-supervised gradient descent learning in the context of the overall brain architecture comprising dynamically coupled perceptual, cognitive, and motor systems [106] that each in turn are organized as epistemically closed.”
BackgroundSequential memory improves sample and memory efficiency in episodic control
Freire, I. T., Amil, A. F., & Verschure, P. F. M. J. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00950-3 · Cited by 6
“Thus far, ERL models do not capitalize on the sequential nature of the temporal unfolding of events that are reflected in the structure of hippocampal memory 21–23 .”
Perkes, I. E., Morris, R. W., Griffiths, K. R., Quail, S. L., Waters, F. A., O’Brien, M., Hazell, P., & Balleine, B. W. (2022) · Biological Psychiatry Global Open Science · doi:10.1016/j.bpsgos.2022.11.004 · Cited by 9
“Nevertheless, the way that these values are integrated with action-outcome retrieval during performance and the neural substrates that support that integrative process are underexplored in humans, as is their causal role in action control (12,13).”
BackgroundDopamine receptors of the rodent fastigial nucleus support skilled reaching for goal-directed action
Caragea, V.-M., Méndez‐Couz, M., & Manahan‐Vaughan, D. (2023) · Brain Structure and Function · doi:10.1007/s00429-023-02685-0 · Cited by 5
“Participation in goal-directed behavior has been ascribed to the cerebellum (Babayan et al. 2017; Verschure et al. 2014) and the deep cerebellar nuclei (DCN) (Callu et al. 2013; Xiao et al. 2018).”
BackgroundSimulated Dopamine Modulation of a Neurorobotic Model of the Basal Ganglia
Prescott, T. J., Montes-González, F., Gurney, K. R., Humphries, M. D., & Redgrave, P. (2024) · Biomimetics · doi:10.3390/biomimetics9030139 · Cited by 5
“With excessively high levels of simulated dopamine, the robot displays hyperactivity and rapid switching between behaviours, symptoms that show some resemblance to hyper-dopaminergic outcomes in animals and humans.”
Toward mechanistic models of action-oriented and detached cognition
Pezzulo, G. (2016) · Behavioral and Brain Sciences · doi:10.1017/s0140525x15001648 · Cited by 5
Machine Morality: From Harm-Avoidance to Human-Robot Cooperation
Freire, I. T., Urikh, D., Arsiwalla, X. D., & Verschure, P. F. M. J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-64313-3_13 · Cited by 5
From motor to visually guided bimanual affordance learning
Sánchez-Fibla, M., Forestier, S., Moulin-Frier, C., Puigbò, J.-Y., & Verschure, P. F. M. J. (2019) · Adaptive Behavior · doi:10.1177/1059712319855836 · Cited by 5
Decoding route selection of pigeon during goal-directed behavior: A joint spike-LFP study
Cheng, S., Li, M., Fan, J., Shang, Z., & Wan, H. (2021) · Behavioural Brain Research · doi:10.1016/j.bbr.2021.113289 · Cited by 5
Esmaeili, A., Antonova, A. M., Sitnikova, E., & Smirnov, K. (2021) · Behavioural Brain Research · doi:10.1016/j.bbr.2021.113627 · Cited by 4
Two possible driving forces supporting the evolution of animal communication
Moulin-Frier, C., & Verschure, P. F. M. J. (2016) · Physics of Life Reviews · doi:10.1016/j.plrev.2016.01.019 · Cited by 4
Fan, M., Liu, Z., Wei, A., Gao, R.-J., Luan, P., & He, X. (2025) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s545022 · Cited by 3
“14 Furthermore, emerging evidence suggests that abnormalities in interoceptive-affective integration may contribute to the development of anomalous self-experiences characteristic of schizophrenia.”
Human foraging strategies flexibly adapt to resource distribution and time constraints
Simonelli, V., Nuzzi, D., Lancia, G. L., & Pezzulo, G. (2025) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-025-01347-4 · Cited by 3
“…that arise during foraging and related tasks have been widely studied in humans and other animals (Cohen et al., 2007; Mehlhorn et al., 2015; Addicott et al., 2017; Daw et al., 2006; Verschure et al., 2014; Maffei et al., 2015; Parr et al., 2022; Pezzulo et al., 2018; Friston et al., 2017).”
Shinya, Y., Oku, K., Watanabe, H., Taga, G., & Fujii, S. (2022) · Experimental Brain Research · doi:10.1007/s00221-022-06379-7 · Cited by 3
“In turn, the physiological needs may enhance auditory-motor learning and goal-directed behavior in young infants (Ecker and Gilead 2018; Nagai 2019; Verschure et al. 2014).”
BackgroundNikrahan, G. R. (2023) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2023.1050164 · Cited by 3
“Xu et al. (2012) found that the quality and stability of marriage until the 40th month of the relationship is predicted by lower activation of medial OFC, right accumbens, and right subcallosal cingulate during the early stages of love (Xu et al., 2012).”
BackgroundInfluentialThe Role of Hp-NCL Network in Goal-Directed Routing Information Encoding of Bird: A Review
Li, M., Shang, Z., Zhao, K., Cheng, S., & Wan, H. (2020) · Brain Sciences · doi:10.3390/brainsci10090617 · Cited by 3
“In this complex behavior, the brain mechanisms of the basic functions of environmental perception and behavioral decision have been widely studied [3].”
BackgroundEffenberg, A. O., & Schmitz, G. H. (2022) · Springer eBooks · doi:10.1007/978-3-662-53386-4_60-1 · Cited by 2
A Model of Agential Learning Using Active Inference
Pitliya, R. J., & Murphy, R. A. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-47958-8_8 · Cited by 2
Neural activity in monkey amygdala during performance of a multisensory operant task
Montes-Lourido, P., Vicente, A. F., Bermúdez, M. A., & González, F. (2015) · Journal of Integrative Neuroscience · doi:10.1142/s021963521550020x · Cited by 3
Metadata
- Authors
- Verschure, P. F. M. J., Pennartz, C. M. A., & Pezzulo, G.
- Affiliations
- Institució Catalana de Recerca i Estudis Avançats (ES)
- University of Amsterdam (NL)
- National Research Council (IT)
- Corresponding author
- Verschure, Paul F. M. J.
- Venue
- Philosophical Transactions of the Royal Society B Biological Sciences
- Publisher
- The Royal Society
- Published
- 2014-11-05
- Volume
- 369
- Issue
- 1655
- Pages
- 20130483
- ISSN
- 0962-8436 (ISSN-L)
- 1471-2970
- PMC
- PMC4186236
- OpenAlex
- W2143590399
- Semantic Scholar
- defe4585fd20817ba121287456fe83112d9582f7
- Cited by
- 131
- References
- 152
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 11 · 2023: 11 · 2022: 9 · 2021: 16 · 2020: 11 · 2019: 13 · 2018: 11 · 2017: 10 · 2016: 15 · 2015: 16 · 2014: 1
- FWCI
- 1.35
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Philosophical Transactions of the Royal Society B Biological Sciences (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion)
- Repositori digital de la UPF (Universitat Pompeu Fabra) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Keywords
- goal-directed behavior, distributed adaptive control, embodied cognition, outcome prediction, valence, basal ganglia, hippocampus, prefrontal cortex, amygdala, hypothalamus, action planning, learning and memory, neurorobotics
- MeSH terms
- Animals, Artificial Intelligence, Behavior, Animal, Brain, Choice Behavior, Cognition, Goals, Rodentia
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
