The DOI matches its Crossref record; 153 of the 153 references it cites were found in a registry or an index.
The why, what, where, when and how of goal-directed choice: neuronal and computational principles
Open accessAbstract
The central problems that goal-directed animals must solve are: 'What do I need and Why, Where and When can this be obtained, and How do I get it?' or the H4W problem. Here, we elucidate the principles underlying the neuronal solutions to H4W using a combination of neurobiological and neurorobotic approaches. First, we analyse H4W from a system-level perspective by mapping its objectives onto the Distributed Adaptive Control embodied cognitive architecture which sees the generation of adaptive action in the real world as the primary task of the brain rather than optimally solving abstract problems. We next map this functional decomposition to the architecture of the rodent brain to test its consistency. Following this approach, we propose that the mammalian brain solves the H4W problem on the basis of multiple kinds of outcome predictions, integrating central representations of needs and drives (e.g. hypothalamus), valence (e.g. amygdala), world, self and task state spaces (e.g. neocortex, hippocampus and prefrontal cortex, respectively) combined with multi-modal selection (e.g. basal ganglia). In our analysis, goal-directed behaviour results from a well-structured architecture in which goals are bootstrapped on the basis of predefined needs, valence and multiple learning, memory and planning mechanisms rather than being generated by a singular computation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1. The DAC theory of mind and brain (see [10] for a review). Left: highly abstract representation of the DAC architecture. DAC proposes that the brain is organized as a three-layered control structure with tight coupling within and between these layers distinguishing: the soma (SL) and the reactive (RL), adaptive (AL) and contextual (CL) layers. Across these layers, a columnar organization exists that deals with the processing of states of the World or exteroception (left, red), the self or interoception (middle, blue) and action (right, green). See text for further explanation. The reactive layer: the RL comprises dedicated behaviour systems (BS) that combine predefined sensorimotor mappings with drive reduction mechanisms that are predicated on the needs of the body (SL). Right lower panel: each BS follows homeostatic principles supporting the self-essential functions (SEF) of the body (SL). In order to map needs into behaviours, the strength of the essential variables served by the BSs, SEFs, have a specific distribution in task-space called an ‘affordance gradient’. In this example, we consider the (internally represented) ‘attractive force’ of the home position supporting the security SEF or of open space defining the exploration SEF. The values of the respective SEFs are defined by the difference between the sensed value of the affordance gradient (red) and its desired value given the prevailing needs (blue). The regulator of each BS defines the next action as to perform a gradient ascent on the SEF. An integration and action selection process across the different BSs forces a strict winner-take-all decision that defines the specific behaviour emitted. The allostatic controller of the RL regulates the internal homeostatic dynamic of the BSs to set priorities defined by needs and environmental opportunities through the modulation of the affordance gradients, desired values of SEFs and/or the integration process. The adaptive layer: the AL acquires a state space of the agent–environment interaction and shapes action. The learning dynamic of AL is constrained by the SEFs of the RL that define value. The AL crucially contributes to exosensing by allowing the processing of states of distal sensors, e.g. vision and audition, which are not predefined but rather are tuned in somatic time to properties of the interaction with the environment. Acquired sensor and motor states are in turn associated through the valence states signalled by the RL. The contextual layer: the core processes of the CL are divided between a task-model and a self-model. The CL expands the time horizon in which the agent can operate through the use of sequential short-term and long-term memory (STM and LTM) systems respectively. These memory systems operate on integrated sensorimotor representations that are generated by the AL and acquire, retain and express goal-oriented action regulated by the RL. The CL comprises a number of processes (right upper panel): (a) when the discrepancy between predicted and encountered sensory states falls below a STM acquisition threshold, the perceptual predictions (red circle) and motor activity (green rectangle) generated by AL are stored in STM as a, so-called, segment. The STM acquisition threshold is defined by the time-averaged reconstruction error of the perceptual learning system of AL. (b) If a goal state (blue flag) is reached, e.g. reward or punishment, the content of STM is retained in LTM as a sequence conserving its order, goal state and valence marker, e.g. aversive or appetitive, and STM is reset. Every sequence is thus labelled with respect to the specific goal it pertains to and its valence marker. (c) If the outputs generated by the RL and AL to action selection are sub-threshold, the AL perceptual predictions are matched against those stored in LTM. (d) The CL selected action is defined as a weighted sum over the segments of LTM. (e) The contribution of LTM segments to decision-making depends on four factors: perceptual evidence, memory chaining, the distance to the goal state and valence. Working memory (WM) of the CL is defined by the memory dynamics that represents these factors. Active segments that contributed to the selected action are associated with those that were previously active establishing rules for future chaining. The self-model component of the CL monitors task performance and develops (re)descriptions of task dynamics anchored in the self. In this way, the system generates meta-representational knowledge that forms autobiographical memory. This aspect of the DAC CL is not further considered in this paper.](/paper-figures/21d6bfd3592c4a3dbfffc67c1ba6b12f/the-why-what-where-when-and-how-of-goal-directed-choice-neuronal-and-figure-1.jpg)
![Figure 2. Proposed brain architecture representing the neuronal substrate of goal-directed behaviour and its relation to the neurorobotic DAC architecture. The hippocampal formation is proposed to code the organism's world state space (red), the prefrontal cortex (especially its medial and orbitofrontal aspects) to represent task space (i.e. rules, constraints, goals and values of cues and action options, purple) and the striatum (and downstream structures of the basal ganglia) to mediate action selection. In this scheme, the hypothalamus and brain stem contain sensor systems monitoring homeostatic variables and providing information about the motivational needs of the organism that define the pursuit of needs and goals (blue). The arrow from striatum to thalamus represents an indirect projection. The hypothalamic efferents are modelled after those traced for the lateral hypothalamus in relation to feeding behaviour and do not apply to hypothalamic areas in general. For the sake of clarity, the scheme's anatomic connections are by no means complete. For instance, outputs from prefrontal cortex and basal ganglia to the brain stem, or several afferent inputs to amygdala and VTA, have not been included, while several hypothalamic nuclei project directly to ventromedial prefrontal areas. Sensory inputs reach the hippocampus via intermediate stations (parahippocampal areas; not shown) and are supplemented with frontal cortical inputs converging on these intermediate areas. Furthermore, the motor cortices are meant to include premotor, supplementary motor and frontal oculomotor areas (based on [40–42]).](/paper-figures/71bcd641609344238bd6e4f3088881f2/the-why-what-where-when-and-how-of-goal-directed-choice-neuronal-and-figure-2.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In most psychological theories, drives are usually linked to basic motivations and homoeostasis (e.g., interoceptive hunger or thirst), while goals are considered to have more elaborate (exteroceptive) sensory and cognitive aspects (e.g., dining in a restaurant), but their relations are often unclear from a mechanistic viewpoint (Pezzulo et al., 2014b, Verschure et al., 2014).”
From Active Inference, homeostatic regulation and adaptive behavioural control (2015) · cites this paper as Verschure et al., 2014
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 1×
“CSL is consistent with ‘predictive brain’ frameworks and ‘free energy minimization’ principles [29,30].”
MethodReal-Time Obstacle Avoidance for Manipulators and Mobile Robots
Khatib, O. (1986) · The International Journal of Robotics Research · doi:10.1177/027836498600500106 · Cited by 7,894 · cited 1×
“This idea has its roots in Gestalt psychology [20] and is a standard approach towards robot navigation [21] but generalizes towards optimal path planning in mobile robots [22] and social interaction in humanoid robots [23].”
BackgroundSearle, J. R. (1980) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00005756 · Cited by 6,954 · cited 1×
“The AL allows the agent to overcome the predefined behavioural repertoire of the RL and to engage an a priori unpredictable world, solving the notorious symbol grounding that led to the demise of classical AI [3,28].”
BackgroundCognitive maps in rats and men
Tolman, E. C. (1948) · Psychological Review · doi:10.1037/h0061626 · Cited by 6,641 · cited 1×
“The first view holds that the hippocampus codes an agent’s position in space, as inspired by the Cognitive Map theory of Tolman [63] as well as the body of place-cell research initiated by O’Keefe & Dostrovsky [64].”
MethodThe hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat
O'Keefe, J., & Dostrovsky, J. (1971) · Brain Research · doi:10.1016/0006-8993(71)90358-1 · Cited by 6,151
Episodic memory: from mind to brain
Tulving, E. (2002) · Annual Review of Psychology · doi:10.1146/annurev.psych.53.100901.135114 · Cited by 4,950 · cited 1×
“The second notion accounts for hippocampal function as a ‘recorder’ of experience—an organ for the formation of episodic memories that would be transferred, with the passage of time, to neocortical sites where also generalization (semanticization) of memory may take place [65].”
BackgroundPredictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564 · cited 1×
“Building on previous models that framed the basal ganglia as an actor–critic architecture for (model-free) reinforcement learning [120,121], we recently argued that especially the ‘Critic’ (goal-predictive) function of the striatum is well supported by the data, whereas its implementation of the ‘Actor’ component is m…”
ResultAffective Neuroscience: The Foundations of Human and Animal Emotions
Panksepp, J. (1998) · doi:10.1093/oso/9780195096736.001.0001 · Cited by 4,428
(1991) · Choice Reviews Online · doi:10.5860/choice.28-5376 · Cited by 4,218
(1998) · Cited by 4,012
Research works printed but not linked: 8
- Verschure PFMJ, 2013, Formal minds and biological brains II: from the mirage of intelligence to a science and engineering of consciousness, IEEE Expert., 28, 33
- Panksepp J, 2012, The archaeology of mind: neuroevolutionary origins of human emotions (Norton series on interpersonal neurobiology)
- Sanchez-Fibla MS, 2010, IEEE/RSJ Int. Conf. 18–22 October 2010
- Miller S, 1928, Sur une forme particuliere des reflexes conditionnels, C.R. Seances Soc. Polonaise Biol., 49, 1155
- Ringwald M, 2007, ECMR 2007–3rd European Conf. on Mobile Robots
- Simerly R, 1995, The rat nervous system, 353
- Gray JA, 2000, The neuropsychology of anxiety: an enquiry into thé functions of thesepto-hippocampal system
- Verschure PFMJ, 2012, The distributed adaptive control architecture of the mind, brain, body nexus, Biol. Inspired Cogn. Archit., 1, 55
Cited by
Sorted by citations
Items 101–131 of 131
Puigbò, J.-Y., Moulin-Frier, C., Vouloutsi, V., Sánchez-Fibla, M., Herreros, I., & Verschure, P. F. M. J. (2015) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2015.7363580 · Cited by 2
“This structure nicely fits the biologically grounded Distributed Adaptive Control framework (DAC, [21], [23]).”
BackgroundPetracca, E. (2022) · Economics and Philosophy · doi:10.1017/s0266267122000189 · Cited by 2
“For instance, cognitive approaches to learning have long maintained that pursuing a goal requires a complex interplay of cognitive processes such as planning and imagination (Verschure et al. 2014).”
BackgroundEffenberg, A. O., & Schmitz, G. H. (2023) · Springer eBooks · doi:10.1007/978-3-662-53410-6_60 · Cited by 2
Grid cells: the missing link in understanding Parkinson’s disease?
Reinshagen, A. (2024) · Frontiers in Neuroscience · doi:10.3389/fnins.2024.1276714 · Cited by 3
“…versa, the striatum and the HF/EC are effectively linked (Finch et al., 1995; Totterdell and Meredith, 1997; Devan and White, 1999; Hartley et al., 2003; Johnson et al., 2007; Lex and Hauber, 2010; van der Meer et al., 2010; Ghiglieri et al., 2011; Verschure et al., 2014; Stoianov et al., 2018).”
BackgroundFeeling Artificial Intelligence. Cognitive Decision-Making Model Borrowed from Living Beings
Kargin, A., & Petrenko, T. (2024) · International Symposium on Networks, Computers and Communications (ISNCC) · doi:10.1109/isncc62547.2024.10759016 · Cited by 1
“Thus, the above two approaches aimed to construct Hybrid AI for robots occupy different niches of hierarchical cognitive architecture [11, 12], and their possible integration in future FAI can high level of the service given by robots.”
Modelling Agency Perception in a Multi-agent Context in Depression Using Active Inference
Pitliya, R. J., Marković, D., Folesani, F., Murri, M. B., Castiello, S., & Murphy, R. A. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-77138-5_2 · Cited by 1
Prescott, T. J., & Jiménez-Rodríguez, A. (2025) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/bs.plm.2025.03.005 · Cited by 1
Li, M., Cheng, S., Fan, J., Shang, Z., & Wan, H. (2022) · Brain Research Bulletin · doi:10.1016/j.brainresbull.2022.03.007 · Cited by 2
Towards sample-efficient policy learning with DAC-ML
Freire, I. T., Amil, A. F., Vouloutsi, V., & Verschure, P. F. M. J. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.06.035 · Cited by 1
“It is built based on the architectural principles of DAC of layered and hierarchical organization, that predicate that cognition is hierarchically distributed in different layers of control [10], [11], [12].”
BackgroundA Generalised Inverse Reinforcement Learning FrameworkPreprint
Jarboui, F., & Perchet, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.11812 · Cited by 2
Goma, S. R. (2015) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2086166
“In other words, in complex designs, the ‘why’ becomes very hard to define as opposed to ‘what-how’ and the ‘why’ is associated with ‘system-level’ view of a problem – in terms of cognition the ‘what-how’ would be problem solving whereas the ‘why’ would define the context in which the problem solving should occur, very…”
ResultKassem, M. S. (2017) · The Sydney eScholarship Repository (The University of Sydney)
Notice of Violation of IEEE Publication Principles: Influence of Emotions in Shaping Decisions
Ghosh, A., & Huang, S. (2019) · IEEE National Aerospace and Electronics Conference (NAECON) · doi:10.1109/naecon46414.2019.9058296
“Continuing along the process of decision-making, experienced emotion can contribute to reinforcement learning [52][53][54][49], increases the memory [55][56] of both positive and negative emotions.”
BackgroundTowards sample-efficient episodic control with DAC-MLPreprint
Freire, I. T., Amil, A. F., Vouloutsi, V., & Verschure, P. F. M. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.13779
“It is built based on the architectural principles of DAC of layered and hierarchical organization, that predicate that cognition is hierarchically distributed in different layers of control [10], [11], [12].”
BackgroundBarros, K. O., de Almeida, D. M., & Bueno, K. D. O. (2021) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
Gaze-Related Activity in Primate Frontal Cortex Predicts and Mitigates Spatial UncertaintyPreprint
Bharmauria, V., Schütz, A., Khoozani, P. A., Yan, X., Wang, H., Bremmer, F., & Crawford, J. D. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.06.20.449147
Computer Aided Search Tasks in a Naturally Occurring Environment
Harrington, K., Pinchin, J., & Sharples, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77932-0_22
Tognoli, E., & Huang, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.01503
Verschure, P. F. M. J. (2021) · Biosystems & biorobotics · doi:10.1007/978-3-030-70316-5_92
Perkes, I. E., Morris, R. W., Griffiths, K. R., Quail, S. L., Waters, F. A., O’Brien, M., Hazell, P., & Balleine, B. W. (2022) · medRxiv · doi:10.1101/2022.03.19.22272645
Borges, V. P. M. (2023) · ResearchEquals · doi:10.53962/m326-qk7j
Kennedy, E. A. (2023) · Digital Repository Service (Northeastern University) · doi:10.17760/d20481307
Schwarb, H. D., Dulas, M. R., Patel, N. N., Bouton, N. A., Cohen, N. J., & Duff, M. C. (2025) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2025.109157
The effects of task-relevant suggestions on motivated cognitive control
Zahedi, A., & Sommer, W. (2025) · International review of neurobiology · doi:10.1016/bs.irn.2025.06.003
Cognitive Architectures: Definition, Examples, and Challenges
Verschure, P. F. M. J. (2024) · Springer eBooks · doi:10.1007/978-3-642-41610-1_206-1 · Cited by 2
The Brain’s Extended Navigation Network
Patai, E. Z., Brown, T. I., Ciocchi, S., Goard, M. J., Jadhav, S. P., Keshavarzi, S., Krupic, J., Laurens, J., Vervaeke, K., & Jeffery, K. J. (2026) · Springer eBooks · doi:10.1007/978-3-032-20563-6_12
Brain-inspired predictive control of robotic sensorimotor systems
Pio‐Lopez, L. (2017) · theses.fr (ABES) · doi:10.70675/cb356dbdz484cz4dd4zaad7z6622007ab33b
Puygrenier, L. (2019) · theses.fr (ABES) · doi:10.70675/c632ab07z5b2ez4f68zaa26z9533043f65ab
Feeling artificial intelligence. Attention engine model borrowed from living beings
Kargin, A., & Petrenko, T. (2024) · International Conference on Information Control Systems & Technologies (ICST) · doi:10.55056/ceur-ws.org/vol-3790/paper39.pdf
“In [10, 11], hypothetical neuronal architecture of the mammalian brain where cognitive processes divided over different layers is proposed.”
Metadata
- Authors
- Verschure, P. F. M. J., Pennartz, C. M. A., & Pezzulo, G.
- Affiliations
- Institució Catalana de Recerca i Estudis Avançats (ES)
- University of Amsterdam (NL)
- National Research Council (IT)
- Corresponding author
- Verschure, Paul F. M. J.
- Venue
- Philosophical Transactions of the Royal Society B Biological Sciences
- Publisher
- The Royal Society
- Published
- 2014-11-05
- Volume
- 369
- Issue
- 1655
- Pages
- 20130483
- ISSN
- 0962-8436 (ISSN-L)
- 1471-2970
- PMC
- PMC4186236
- OpenAlex
- W2143590399
- Semantic Scholar
- defe4585fd20817ba121287456fe83112d9582f7
- Cited by
- 131
- References
- 152
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 11 · 2023: 11 · 2022: 9 · 2021: 16 · 2020: 11 · 2019: 13 · 2018: 11 · 2017: 10 · 2016: 15 · 2015: 16 · 2014: 1
- FWCI
- 1.35
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Philosophical Transactions of the Royal Society B Biological Sciences (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion)
- Repositori digital de la UPF (Universitat Pompeu Fabra) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Keywords
- goal-directed behavior, distributed adaptive control, embodied cognition, outcome prediction, valence, basal ganglia, hippocampus, prefrontal cortex, amygdala, hypothalamus, action planning, learning and memory, neurorobotics
- MeSH terms
- Animals, Artificial Intelligence, Behavior, Animal, Brain, Choice Behavior, Cognition, Goals, Rodentia
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
