The why, what, where, when and how of goal-directed choice: neuronal and computational principles
Open accessAbstract
The central problems that goal-directed animals must solve are: 'What do I need and Why, Where and When can this be obtained, and How do I get it?' or the H4W problem. Here, we elucidate the principles underlying the neuronal solutions to H4W using a combination of neurobiological and neurorobotic approaches. First, we analyse H4W from a system-level perspective by mapping its objectives onto the Distributed Adaptive Control embodied cognitive architecture which sees the generation of adaptive action in the real world as the primary task of the brain rather than optimally solving abstract problems. We next map this functional decomposition to the architecture of the rodent brain to test its consistency. Following this approach, we propose that the mammalian brain solves the H4W problem on the basis of multiple kinds of outcome predictions, integrating central representations of needs and drives (e.g. hypothalamus), valence (e.g. amygdala), world, self and task state spaces (e.g. neocortex, hippocampus and prefrontal cortex, respectively) combined with multi-modal selection (e.g. basal ganglia). In our analysis, goal-directed behaviour results from a well-structured architecture in which goals are bootstrapped on the basis of predefined needs, valence and multiple learning, memory and planning mechanisms rather than being generated by a singular computation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In most psychological theories, drives are linked to basic motivations and homoeostasis, while goals are considered to have more elaborate exteroceptive sensory and cognitive aspects, but their relations are often unclear from a mechanistic viewpoint.
“In most psychological theories, drives are usually linked to basic motivations and homoeostasis (e.g., interoceptive hunger or thirst), while goals are considered to have more elaborate (exteroceptive) sensory and cognitive aspects (e.g., dining in a restaurant), but their relations are often unclear from a mechanistic viewpoint (Pezzulo et al., 2014b, Verschure et al., 2014).”
From Active Inference, homeostatic regulation and adaptive behavioural control (2015) · cites this paper as Verschure et al., 2014
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 1× · see citing works
“CSL is consistent with ‘predictive brain’ frameworks and ‘free energy minimization’ principles [29,30].”
MethodReal-Time Obstacle Avoidance for Manipulators and Mobile Robots
Khatib, O. (1986) · The International Journal of Robotics Research · doi:10.1177/027836498600500106 · Cited by 7,894 · cited 1× · see citing works
“This idea has its roots in Gestalt psychology [20] and is a standard approach towards robot navigation [21] but generalizes towards optimal path planning in mobile robots [22] and social interaction in humanoid robots [23].”
BackgroundSearle, J. R. (1980) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00005756 · Cited by 6,954 · cited 1× · see citing works
“The AL allows the agent to overcome the predefined behavioural repertoire of the RL and to engage an a priori unpredictable world, solving the notorious symbol grounding that led to the demise of classical AI [3,28].”
BackgroundCognitive maps in rats and men
Tolman, E. C. (1948) · Psychological Review · doi:10.1037/h0061626 · Cited by 6,641 · cited 1× · see citing works
“The first view holds that the hippocampus codes an agent’s position in space, as inspired by the Cognitive Map theory of Tolman [63] as well as the body of place-cell research initiated by O’Keefe & Dostrovsky [64].”
MethodThe hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat
O'Keefe, J., & Dostrovsky, J. (1971) · Brain Research · doi:10.1016/0006-8993(71)90358-1 · Cited by 6,151 · see citing works
Episodic memory: from mind to brain
Tulving, E. (2002) · Annual Review of Psychology · doi:10.1146/annurev.psych.53.100901.135114 · Cited by 4,950 · cited 1× · see citing works
“The second notion accounts for hippocampal function as a ‘recorder’ of experience—an organ for the formation of episodic memories that would be transferred, with the passage of time, to neocortical sites where also generalization (semanticization) of memory may take place [65].”
BackgroundPredictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564 · cited 1× · see citing works
“Building on previous models that framed the basal ganglia as an actor–critic architecture for (model-free) reinforcement learning [120,121], we recently argued that especially the ‘Critic’ (goal-predictive) function of the striatum is well supported by the data, whereas its implementation of the ‘Actor’ component is m…”
ResultAffective Neuroscience: The Foundations of Human and Animal Emotions
Panksepp, J. (1998) · doi:10.1093/oso/9780195096736.001.0001 · Cited by 4,428 · see citing works
(1991) · Choice Reviews Online · doi:10.5860/choice.28-5376 · Cited by 4,217 · see citing works
(1998) · Cited by 4,012 · see citing works
Principles of topological psychology.
Lewin, K. (1936) · McGraw-Hill eBooks · doi:10.1037/10019-000 · Cited by 3,967 · see citing works
The psychology of fear and stress
(1988) · Choice Reviews Online · doi:10.5860/choice.26-2399 · Cited by 3,134 · cited 1× · see citing works
“These sensorimotor loops are organized in fundamental and opposing behaviour systems that support basic needs such as fight, flight [15], seek and play [16].”
BackgroundReactivation of hippocampal ensemble memories during sleep
Wilson, M. A., & McNaughton, B. L. (1994) · Science · doi:10.1126/science.8036517 · Cited by 3,118 · cited 1× · see citing works
“In addition to storing information, the hippocampus is able to retrieve previously stored information and to self-generate internal sequences of cell activity that are subsequently used to map novel environments or situations [82–84].”
Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex.
Pavlov, P. I. (1930) · Archives of Neurology And Psychiatry · doi:10.1001/archneurpsyc.1930.02220110256018 · Cited by 2,681 · see citing works
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 1× · see citing works
“Neuroscience and neuroeconomics have often benefited from a close linkage to computational methods but these often abstract from many details of situated action (e.g. model-based reinforcement learning theories [134,135] and several contributions to this special issue).”
BackgroundPurposive Behavior in Animals and Men
Rodhom, C., & Tolman, E. C. (1950) · The American Journal of Psychology · doi:10.2307/1418946 · Cited by 2,455 · cited 1× · see citing works
“With the cognitive revolution, these notions were again incorporated in the terminology of mind and brain [1] but the mechanistic functioning and neuronal underpinnings of goal-directed behaviour remained largely unknown.”
BackgroundAlexander, G. E., Crutcher, M. D., & DeLong, M. R. (1990) · PubMed · Cited by 2,422 · see citing works
The role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402 · cited 1× · see citing works
“associations (and ensuing habits) can be acquired via the dorsal striatal system [80,81].”
Background“Materialism and Qualia: The Explanatory Gap
Levine, J. (1983) · Pacific Philosophical Quarterly · doi:10.1111/j.1468-0114.1983.tb00207.x · Cited by 2,150 · cited 1× · see citing works
“As much as there is a ‘hard problem’ in the explanation of consciousness [4,5], we can argue that a similar explanatory gap exists in understanding goal-oriented behaviour, it also critically depends on inferences by the observer on the beliefs an organism entertains with respect to its goals and tasks.”
BackgroundFacing up to the problem of consciousness
Chalmers, D. J. (1996) · Toward a Science of Consciousness · doi:10.7551/mitpress/6860.003.0003 · Cited by 1,966 · see citing works
(1991) · Progress in Brain Research · doi:10.1016/s0079-6123(08)62678-3 · Cited by 1,961 · see citing works
Conditioned Reflexes: an Investigation of the Physiological Activity of the Cerebral Cortex
Denny-Brown, D. E. (1928) · Nature · doi:10.1038/121662a0 · Cited by 1,854 · see citing works
Neural correlates of decision variables in parietal cortex
Platt, M. L., & Glimcher, P. W. (1999) · Nature · doi:10.1038/22268 · Cited by 1,721 · cited 1× · see citing works
“Even structures such as posterior parietal cortex and primary sensory cortex (A1, V1) are deeply affected by associative stimulus–reward and action–reward learning [56–62].”
BackgroundPattern Separation in the Dentate Gyrus and CA3 of the Hippocampus
Leutgeb, J. K., Leutgeb, S., Moser, M.-B., & Moser, E. I. (2007) · Science · doi:10.1126/science.1135801 · Cited by 1,601 · see citing works
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598 · cited 1× · see citing works
“Importantly, orbito-frontal and medial prefrontal-anterior cingulate neurons are sensitive to the motivational value of cues [100–104] and actions associated with goal pursuit [105,106].”
BackgroundThe effects of changes in the environment on the spatial firing of hippocampal complex-spike cells
Muller, R., & Kubie, J. (1987) · The Journal of Neuroscience · doi:10.1523/jneurosci.07-07-01951.1987 · Cited by 1,581 · cited 1× · see citing works
“Moreover, the representation of the task seems to follow a multiplexing of input streams combining sensory, location and action information at both the input and memory stages of hippocampal processing wood [66,71–73].”
BackgroundGoal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541 · cited 3× · see citing works
“Furthermore, if the value/utility of a goal changes, for instance because of a change in motivational state, the organism should refrain from persistently conducting a fixed stimulus–response sequence or habit, but must adjust its response according to the change in value [7].”
BackgroundDissociation in prefrontal cortex of affective and attentional shifts
Dias, R., Robbins, T. W., & Roberts, A. C. (1996) · Nature · doi:10.1038/380069a0 · Cited by 1,527 · cited 1× · see citing works
“Lesion studies have confirmed a causal involvement of prefrontal structures in representing goals and task rules, implying orbitofrontal cortex in reversal learning and medial prefrontal cortex in both extra-dimensional shifting [107,108] and learning action–outcome relationships [7,109].”
BackgroundPackard, M. G., & McGaugh, J. L. (1996) · Neurobiology of Learning and Memory · doi:10.1006/nlme.1996.0007 · Cited by 1,505 · cited 1× · see citing works
“B 369 : 20130483 associations (and ensuing habits) can be acquired via the dorsal striatal system [80,81].”
BackgroundConditioned reflexes: An investigation of the physiological activity of the cerebral cortex
Pavlov, I. P. (2010) · Annals of Neurosciences · doi:10.5214/ans.0972-7531.1017309 · Cited by 1,466 · cited 1× · see citing works
“…by the RL, following the paradigm of classical conditioning where initially neutral or conditioned stimuli (CS) obtain the ability to trigger actions, or conditioned reponses (CR), by virtue of their contingent presentation with intrinsically motivational stimuli or unconditioned stimuli (US) [12].”
BackgroundAllostasis: A New Paradigm to Explain Arousal Pathology
Sterling, P., & Eyer, J. (1988) · Zenodo · doi:10.5281/zenodo.13537700 · Cited by 1,450 · see citing works
Medial Frontal Cortex Mediates Perceptual Attentional Set Shifting in the Rat
Birrell, J. M., & Brown, V. J. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-11-04320.2000 · Cited by 1,421 · cited 1× · see citing works
“Lesion studies have confirmed a causal involvement of prefrontal structures in representing goals and task rules, implying orbitofrontal cortex in reversal learning and medial prefrontal cortex in both extra-dimensional shifting [107,108] and learning action–outcome relationships [7,109].”
BackgroundMultiple dopamine functions at different time courses
Schultz, W. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135722 · Cited by 1,415 · cited 1× · see citing works
“…such as comparison—at least in the domain of reward and appetitive learning—is the reward-prediction-error mechanism that may be implemented by mesodiencephalic dopaminergic neurons projecting to striatum and prefrontal cortex [125], while the habenula has been implied in aversive learning [126].”
BackgroundFacing Up to the Problem of Consciousness
Chalmers, D. J. (1998) · Routledge eBooks · doi:10.4324/9780203826430-11 · Cited by 1,334 · see citing works
Relative reward preference in primate orbitofrontal cortex
Tremblay, L., & Schultz, W. (1999) · Nature · doi:10.1038/19525 · Cited by 1,314 · cited 1× · see citing works
“Importantly, orbitofrontal and medial prefrontal-anterior cingulate neurons are sensitive to the motivational value of cues [100–104] and actions associated with goal pursuit [105,106].”
BackgroundInternally Generated Cell Assembly Sequences in the Rat Hippocampus
Pastalkova, E., Itskov, V., Amarasingham, A., & Buzsáki, G. (2008) · Science · doi:10.1126/science.1159775 · Cited by 1,301 · see citing works
Hippocampal place-cell sequences depict future paths to remembered goals
Pfeiffer, B. E., & Foster, D. J. (2013) · Nature · doi:10.1038/nature12112 · Cited by 1,216 · cited 3× · see citing works
“Thus, during ongoing goal-directed behaviour, as well as ‘off-line’, the hippocampus has multiple modes to (re-)generate and recall information from memory, which can be flexibly used to guide decision-making and/or support consolidation [90,91].”
BackgroundThe basal ganglia: a vertebrate solution to the selection problem?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Neuroscience · doi:10.1016/s0306-4522(98)00319-4 · Cited by 1,199 · see citing works
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190 · see citing works
Putting a spin on the dorsal-ventral divide of the striatum
Voorn, P., Vanderschuren, L. J. M. J., Groenewegen, H. J., Robbins, T. W., & Pennartz, C. M. A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.06.006 · Cited by 1,188 · cited 1× · see citing works
“…inputs), dorsomedial sector (mainly anterior cingulate and prelimbic inputs), a ventrolateral sector (the‘core’ ofnucleusaccumbens; mainlyamygdaloid, prelimbic and dorsal hippocampal input) and a ventromedial sector (‘shell’; mainly prelimbic, infralimbic and ventral hippocampal input [115]).”
BackgroundGläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166 · see citing works
The role of the dorsomedial striatum in instrumental conditioning
Yin, H. H., Ostlund, S. B., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04218.x · Cited by 1,099 · cited 1× · see citing works
“The dorsomedial striatum, for instance, has been implied in action-outcome learning [122], whereas the ventral striatal core functions in cue-outcome learning, in conjunction with its strong amygda-loid input [9,123].”
BackgroundSingle neurons in prefrontal cortex encode abstract rules
Wallis, J. D., Anderson, K. C., & Miller, E. K. (2001) · Nature · doi:10.1038/35082081 · Cited by 1,075 · cited 1× · see citing works
“Neurophysiological studies in rodents and primates indicate that prefrontal neurons can encode task rules that need to be followed to obtain a goal [94], individual actions or chunks of actions leading up to a goal [95–97], and goals and goal sites themselves [98,99].”
BackgroundThe Hippocampus, Memory, and Place Cells
Eichenbaum, H., Dudchenko, P., Wood, E., Shapiro, M., & Tanila, H. (1999) · Neuron · doi:10.1016/s0896-6273(00)80773-4 · Cited by 1,064 · see citing works
Theta rhythms coordinate hippocampal-prefrontal interactions in a spatial memory task
Jones, M. W., & Wilson, M. A. (2005) · PLoS Biology · doi:10.1371/journal.pbio.0030402 · Cited by 1,036 · see citing works
Matching Behavior and the Representation of Value in the Parietal Cortex
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004) · Science · doi:10.1126/science.1094765 · Cited by 1,030 · cited 2× · see citing works
“Even structures such as posterior parietal cortex and primary sensory cortex (A1, V1) are deeply affected by associative stimulus–reward and action–reward learning [56–62].”
BackgroundMethodNeural ensembles in CA3 transiently encode paths forward of the animal at a decision point
Johnson, A., & Redish, A. D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3761-07.2007 · Cited by 1,003 · see citing works
Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval
Carr, M. F., Jadhav, S. P., & Frank, L. M. (2011) · Nature Neuroscience · doi:10.1038/nn.2732 · Cited by 958 · cited 1× · see citing works
“In addition to storing information, the hippocampus is able to retrieve previously stored information and to self-generate internal sequences of cell activity that are subsequently used to map novel environments or situations [82–84].”
The habenula: from stress evasion to value-based decision-making
Hikosaka, O. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2866 · Cited by 955 · cited 1× · see citing works
“…such as comparison—at least in the domain of reward and appetitive learning—is the reward-prediction-error mechanism that may be implemented by mesodiencephalic dopaminergic neurons projecting to striatum and prefrontal cortex [125], while the habenula has been implied in aversive learning [126].”
BackgroundAwake hippocampal sharp-wave ripples support spatial memory
Jadhav, S. P., Kemere, C., German, P. W., & Frank, L. M. (2012) · Science · doi:10.1126/science.1217230 · Cited by 924 · cited 2× · see citing works
“…retrieval is thought to take place as ‘replay’ mainly occurring during hippocampal local-field potential (LFP) events called sharp-wave ripples [85–88], but also during bouts of LFP theta-band activity during running and other ongoing behaviour, resulting in ‘forward sweeps’ of hippocampal place…”
BackgroundFunctional identification of an aggression locus in the mouse hypothalamus
Lin, D., Boyle, M. P., Dollar, P., Lee, H., Lein, E. S., Perona, P., & Anderson, D. J. (2011) · Nature · doi:10.1038/nature09736 · Cited by 914 · cited 1× · see citing works
“For goal-directed behaviour, a first point to note is that drives and their associated behavioural expression, such as aggression, thirst, hunger, sex, sleep and safety, are determined in basic form at the level of the hypothalamus together with lower-order structures [50], analogous to the functioning of the RL in DA…”
BackgroundIndependent Codes for Spatial and Episodic Memory in Hippocampal Neuronal Ensembles
Leutgeb, S., Leutgeb, J. K., Barnes, C. A., Moser, E. I., McNaughton, B. L., & Moser, M. (2005) · Science · doi:10.1126/science.1114037 · Cited by 911 · cited 1× · see citing works
“neutral environmental cues such as odours, and cues that predict reward, as well as the time spent in a situation conducive to goal pursuit [66–70].”
BackgroundAlexander, G. E. (1998) · Cited by 891 · see citing works
Vinkhuyzen, R. E., Verschure, P. F. M. J., & Newell, A. (1994) · The American Journal of Psychology · doi:10.2307/1422886 · Cited by 831 · see citing works
Neural synchrony in cortical networks: history, concept and current status
Uhlhaas, P. (2009) · Frontiers in Integrative Neuroscience · doi:10.3389/neuro.07.017.2009 · Cited by 758 · see citing works
Kudrimoti, H. S., Barnes, C. A., & McNaughton, B. L. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-10-04090.1999 · Cited by 753 · cited 1× · see citing works
“Self-initiated retrieval is thought to take place as ‘replay’ mainly occurring during hippocampal local-field potential (LFP) events called sharp-wave ripples [85–88], but also during bouts of LFP theta-band activity during running and other ongoing behaviour, resulting in ‘forward sweeps’ of hippocampal place represe…”
BackgroundPreplay of future place cell sequences by hippocampal cellular assemblies
Dragoi, G., & Tonegawa, S. (2011) · Nature · doi:10.1038/nature09633 · Cited by 716 · see citing works
The global record of memory in hippocampal neuronal activity
Wood, E. R., Dudchenko, P. A., & Eichenbaum, H. (1999) · Nature · doi:10.1038/17605 · Cited by 707 · cited 2× · see citing works
“neutral environmental cues such as odours, and cues that predict reward, as well as the time spent in a situation conducive to goal pursuit [66–70].”
BackgroundCentral mechanisms of osmosensation and systemic osmoregulation
Bourque, C. W. (2008) · Nature reviews. Neuroscience · doi:10.1038/nrn2400 · Cited by 703 · see citing works
Decision Making as a Window on Cognition
Shadlen, M. N., & Kiani, R. (2013) · Neuron · doi:10.1016/j.neuron.2013.10.047 · Cited by 660 · see citing works
Reward Timing in the Primary Visual Cortex
Shuler, M. G. H., & Bear, M. F. (2006) · Science · doi:10.1126/science.1123513 · Cited by 650 · cited 1× · see citing works
“Even structures such as posterior parietal cortex and primary sensory cortex (A1, V1) are deeply affected by associative stimulus–reward and action–reward learning [56–62].”
BackgroundMotivation and cognitive control in the human prefrontal cortex
Kouneiher, F., Charron, S., & Koechlin, E. (2009) · Nature Neuroscience · doi:10.1038/nn.2321 · Cited by 590 · cited 1× · see citing works
“…and events relevant for pursuing a goal in space and time (What, When and Where), the prefrontal cortex appears better equipped to represent a task space, i.e. the set of rules, constraints, goals and goal-predictive values of cues and actions available as options to pursue goals (How) (cf. [110]).”
BackgroundFast-Forward Playback of Recent Memory Sequences in Prefrontal Cortex During Sleep
Euston, D. R., Tatsuno, M., & McNaughton, B. L. (2007) · Science · doi:10.1126/science.1148979 · Cited by 586 · cited 1× · see citing works
“How hippocampal events are precisely linked to prefrontal processing is less clear yet, but the medial prefrontal cortex does exhibit replay, and its activity coheres with hippocampal theta activity [132,133].”
BackgroundHippocampal “Time Cells”: Time versus Path Integration
Kraus, B. J., Robinson, R. J., White, J. A., Eichenbaum, H., & Hasselmo, M. E. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.015 · Cited by 526 · cited 1× · see citing works
“neutral environmental cues such as odours, and cues that predict reward, as well as the time spent in a situation conducive to goal pursuit [66–70].”
BackgroundSchoenbaum, G., Setlow, B., Saddoris, M. P., & Gallagher, M. (2003) · Neuron · doi:10.1016/s0896-6273(03)00474-4 · Cited by 484 · see citing works
Adaptive Critics and the Basal Ganglia
Barto, A. G. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0018 · Cited by 466 · see citing works
Neurons in the frontal lobe encode the value of multiple decision variables
Kennerley, S. W., Dahmubed, A. F., Lara, A. H., & Wallis, J. D. (2009) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21100 · Cited by 453 · cited 1× · see citing works
“Importantly, orbito-frontal and medial prefrontal-anterior cingulate neurons are sensitive to the motivational value of cues [100–104] and actions associated with goal pursuit [105,106].”
BackgroundThe prefrontal cortex and the integration of sensory, limbic and autonomic information
Groenewegen, H. J., & Uylings, H. B. M. (2000) · Progress in brain research · doi:10.1016/s0079-6123(00)26003-2 · Cited by 442 · cited 1× · see citing works
“Furthermore, the motor cortices are meant to include premotor, supplementary motor and frontal oculomotor areas (based on [40–42]).”
BackgroundCentral control of thermogenesis in mammals
Morrison, S. F., Nakamura, K., & Madden, C. J. (2008) · Experimental Physiology · doi:10.1113/expphysiol.2007.041848 · Cited by 424 · cited 1× · see citing works
“For instance, the preoptic area contains thermosensitive neurons that, at the same time, are involved in the generation and dissipation of body heat [43].”
BackgroundThe hypocretins: Setting the arousal threshold
Sutcliffe, J. G., & de Lecea, L. (2002) · Nature reviews. Neuroscience · doi:10.1038/nrn808 · Cited by 416 · cited 1× · see citing works
“[45–49]).”
Hippocampus leads ventral striatum in replay of place-reward information
Lansink, C. S., Goltstein, P. M., Lankelma, J. V., McNaughton, B. L., & Pennartz, C. M. A. (2009) · PLoS Biology · doi:10.1371/journal.pbio.1000173 · Cited by 400 · see citing works
The Neurobiology of the Prefrontal Cortex: Anatomy, Evolution, and the Origin of Insight
Passingham, R. E., & Wise, S. P. (2012) · doi:10.1093/acprof:osobl/9780199552917.001.0001 · Cited by 379 · see citing works
Beyond the Cognitive Map: From Place Cells to Episodic Memory
Redish, A. D. (1999) · Cited by 369 · see citing works
The effect of lesions of the basolateral amygdala on instrumental conditioning
Balleine, B. W., Killcross, A. S., & Dickinson, A. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-02-00666.2003 · Cited by 355 · see citing works
Ciano, P. D., & Everitt, B. J. (2004) · Journal of Neuroscience · doi:10.1523/jneurosci.1581-04.2004 · Cited by 333 · cited 1× · see citing works
“The dorsomedial striatum, for instance, has been implied in action-outcome learning [122], whereas the ventral striatal core functions in cue-outcome learning, in conjunction with its strong amygdaloid input [9,123].”
BackgroundOrbitofrontal Cortex Supports Behavior and Learning Using Inferred But Not Cached Values
Jones, J. L., Esber, G. R., McDannald, M. A., Gruber, A. J., Hernández, A., Mirenzi, A., & Schoenbaum, G. M. (2012) · Science · doi:10.1126/science.1227489 · Cited by 332 · see citing works
Hypothalamic nutrient sensing in the control of energy homeostasis
Blouet, C., & Schwartz, G. J. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2009.12.024 · Cited by 321 · cited 1× · see citing works
“[45–49]).”
Inhibitory Interactions Between Spiny Projection Neurons in the Rat Striatum
Tunstall, M. J., Oorschot, D. E., Kean, A., & Wickens, J. R. (2002) · Journal of Neurophysiology · doi:10.1152/jn.2002.88.3.1263 · Cited by 311 · cited 1× · see citing works
“However, striatal principal cells are connected via GABAergic recurrent collaterals, providing a potential mechanism for competitive selection [117,118].”
BackgroundRedish, A. D. (1999) · The MIT Press eBooks · doi:10.7551/mitpress/1571.001.0001 · Cited by 310 · cited 1× · see citing works
“This study also illustrates the relationship between taxon versus route navigation strategies [39] in a robotic system.”
BackgroundThe Neurobiology of the Prefrontal Cortex: Anatomy, Evolution, and the Origin of Insight
Passingham, R. E., & Wise, S. P. (2012) · Cited by 307 · see citing works
Stefan Schaal, A., Ijspeert, A. J., Billard, A., Vijayakumar, S., Hallam, J. C. T., & Meyer, J.-A. (2004) · University of Southern Denmark Research Portal (University of Southern Denmark) · Cited by 300 · see citing works
Plasticity of temporal information processing in the primary auditory cortex
Kilgard, M. P., & Merzenich, M. M. (1998) · Nature Neuroscience · doi:10.1038/3729 · Cited by 298 · cited 1× · see citing works
“Even structures such as posterior parietal cortex and primary sensory cortex (A1, V1) are deeply affected by associative stimulus–reward and action–reward learning [56–62].”
BackgroundCovert expectation-of-reward in rat ventral striatum at decision points
van der Meer, M. A. A. (2009) · Frontiers in Integrative Neuroscience · doi:10.3389/neuro.07.001.2009 · Cited by 296 · see citing works
The anatomy of choice: dopamine and decision-making.
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0481 · Cited by 289 · cited 1× · see citing works
“CSL is consistent with ‘predictive brain’ frameworks and ‘free energy minimization’ principles [29,30].”
MethodThe hippocampal-striatal axis in learning, prediction and goal-directed behavior
Pennartz, C. M. A., Ito, R., Verschure, P. F. M. J., Battaglia, F. P., & Robbins, T. W. (2011) · Trends in Neurosciences · doi:10.1016/j.tins.2011.08.001 · Cited by 287 · cited 5× · see citing works
“Error signals incorporating model-based, highdimensional information (‘I got a banana instead of the apple I expected, but they are worth the same’) may involve cortical systems such as the anterior cingulate and orbitofrontal cortex, but also striatal regions [9,127,128] (see [31] for a more detailed computational an…”
BackgroundResultInfluentialCoding for spatial goals in the prelimbic/infralimbic area of the rat frontal cortex
Hok, V., Save, É., Lenck‐Santini, P., & Poucet, B. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0407332102 · Cited by 269 · cited 1× · see citing works
“Neurophysiological studies in rodents and primates indicate that prefrontal neurons can encode task rules that need to be followed to obtain a goal [94], individual actions or chunks of actions leading up to a goal [95–97], and goals and goal sites themselves [98,99].”
BackgroundMind versus metabolism in the control of food intake and energy balance
Berthoud, H. (2004) · Physiology & Behavior · doi:10.1016/j.physbeh.2004.04.034 · Cited by 259 · cited 2× · see citing works
“Furthermore, the motor cortices are meant to include premotor, supplementary motor and frontal oculomotor areas (based on [40–42]).”
BackgroundPurposive Behavior in Animals and Man
Young, P. T., & Tolman, E. C. (1933) · The American Journal of Psychology · doi:10.2307/1414208 · Cited by 249 · cited 1× · see citing works
“With the cognitive revolution, these notions were again incorporated in the terminology of mind and brain [1] but the mechanistic functioning and neuronal underpinnings of goal-directed behaviour remained largely unknown.”
BackgroundConcepts and models of sleep regulation: an overview
Borbély, A. A., & Achermann, P. (1992) · Journal of Sleep Research · doi:10.1111/j.1365-2869.1992.tb00013.x · Cited by 244 · cited 1× · see citing works
“[18,51]).”
Ito, R., Robbins, T. W., Pennartz, C. M. A., & Everitt, B. J. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1615-08.2008 · Cited by 230 · see citing works
Davis, K. L., & Panksepp, J. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2011.04.004 · Cited by 227 · see citing works
Is a bird in the hand worth two in the future? The neuroeconomics of intertemporal decision-making
Tobias Kalenscher, A., & Pennartz, C. M. A. (2007) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2007.11.004 · Cited by 222 · cited 1× · see citing works
“model-based reinforcement learning theories [134,135] and several contributions to this special issue).”
BackgroundInternally generated sequences in learning and executing goal-directed behavior
Pezzulo, G., van der Meer, M. A. A., Lansink, C. S., & Pennartz, C. M. A. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.06.011 · Cited by 217 · cited 1× · see citing works
“Thus, during ongoing goal-directed behaviour, as well as ‘off-line’, the hippocampus has multiple modes to (re-)generate and recall information from memory, which can be flexibly used to guide decision-making and/or support consolidation [90,91].”
BackgroundSelf-citationEnvironmentally mediated synergy between perception and behaviour in mobile robots
Verschure, P. F. M. J., Voegtlin, T., & Douglas, R. J. (2003) · Nature · doi:10.1038/nature02024 · Cited by 216 · cited 2× · see citing works
“non-neuronal feedback resulting from both the high spatio-temporal correlation in sensory streams combined with experience-dependent biased sampling of the sensory space [13,24].”
BackgroundOn the challenges and mechanisms of embodied decisions
Cisek, P., & Pastor‐Bernier, A. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0479 · Cited by 196 · cited 1× · see citing works
“In this integration, we take as our specific question whether goals should be seen as single scalar functions against which behaviour is optimized or whether they are multi-dimensional processes that emerge from the interaction across a number of perceptual, affective, cognitive and motor systems (see also [6]).”
BackgroundWoody, E. Z., & Szechtman, H. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.08.003 · Cited by 195 · cited 1× · see citing works
“[45–49]).”
The algorithmic anatomy of model-based evaluation
Daw, N. D., & Dayan, P. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0478 · Cited by 195 · cited 2× · see citing works
“Third, the strict distinction between retrospective model-free and prospective model-based decision-making is misleading in the sense that in both cases, the state space on which decision-making operates is defined based on models albeit of varying spatio-temporal complexity [31].”
BackgroundThe Mechanics of Embodiment: A Dialog on Embodiment and Computational Modeling
Pezzulo, G., Barsalou, L. W., Cangelosi, A., Fischer, M. H., McRae, K., & Spivey, M. J. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00005 · Cited by 193 · cited 1× · see citing works
“…neurophysiological and neurorobotic approach holds the promise to simultaneously tackle the problem of goal-directed choice at multiple levels [140–142]: (i) the func-tional-behavioural level, e.g. how goals are selected and realized in the real world based on their expected value and the cost of…”
BackgroundUbiquity and Specificity of Reinforcement Signals throughout the Human Brain
Vickery, T. J., Chun, M. M., & Lee, D. (2011) · Neuron · doi:10.1016/j.neuron.2011.08.011 · Cited by 185 · cited 1× · see citing works
“Even structures such as posterior parietal cortex and primary sensory cortex (A1, V1) are deeply affected by associative stimulus–reward and action–reward learning [56–62].”
BackgroundLearning-induced changes of auditory receptive fields
Weinberger, N. M. (1993) · Current Opinion in Neurobiology · doi:10.1016/0959-4388(93)90058-7 · Cited by 166 · cited 1× · see citing works
“Even structures such as posterior parietal cortex and primary sensory cortex (A1, V1) are deeply affected by associative stimulus–reward and action–reward learning [56–62].”
BackgroundPezzulo, G., Rigoli, F., & Chersi, F. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00092 · Cited by 163 · see citing works
Self-citationAnatomical Substrates of Hypothalamic Integration
Simerly, R. B. (2004) · Elsevier eBooks · doi:10.1016/b978-012547638-6/50015-8 · Cited by 148 · see citing works
Thinking as the control of imagination: a conceptual framework for goal-directed systems
Pezzulo, G., & Castelfranchi, C. (2009) · Psychological Research Psychologische Forschung · doi:10.1007/s00426-009-0237-z · Cited by 141 · cited 1× · see citing works
“However, through the state space learning systems of the AL and the task learning systems of the CL, these goals become incrementally more abstracted from their homeostatic origins (see also [137,138]).”
BackgroundActivity in prefrontal cortex during dynamic selection of action sequences
Averbeck, B. B., Sohn, J.-W., & Lee, D. (2006) · Nature Neuroscience · doi:10.1038/nn1634 · Cited by 141 · see citing works
Rondi‐Reig, L., Petit, G. H., Tobin, C., Tonegawa, S., Mariani, J., & Berthoz, A. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.3408-05.2006 · Cited by 139 · cited 1× · see citing works
“The update on the second notion of the recording of experience is that the hippocampus has turned out not only to record (i.e. encode and store) spatial experiences, but also—and more generally—chains of associated events and sequences of motor actions [77–79].”
BackgroundStolzenberg, D. S., & Numan, M. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.10.003 · Cited by 137 · see citing works
Distributed adaptive control: The self-organization of structured behavior
Verschure, P. F. M. J., Kröse, B., & Pfeifer, R. (1992) · Robotics and Autonomous Systems · doi:10.1016/0921-8890(92)90054-3 · Cited by 135 · cited 1× · see citing works
“The reactive layer (RL) of DAC supports the basic functionality of the SL and generates control signals that drive and modulate the engagement of higher control layers and their epistemic functions [14] (figure 1, lower right panel).”
BackgroundTaverna, S., van Dongen, Y. C., Groenewegen, H. J., & Pennartz, C. M. A. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00892.2003 · Cited by 130 · see citing works
Single-Neuron Mechanisms Underlying Cost-Benefit Analysis in Frontal Cortex
Hosokawa, T., Kennerley, S. W., Sloan, J., & Wallis, J. D. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.2221-13.2013 · Cited by 128 · cited 1× · see citing works
“Importantly, orbito-frontal and medial prefrontal-anterior cingulate neurons are sensitive to the motivational value of cues [100–104] and actions associated with goal pursuit [105,106].”
BackgroundFrontal units of the monkey coding the associative significance of visual and auditory stimuli
Watanabe, M. (1992) · Experimental Brain Research · doi:10.1007/bf00228241 · Cited by 123 · see citing works
Cabral, H., Vinck, M. A., Fouquet, C., Pennartz, C. M. A., Rondi‐Reig, L., & Battaglia, F. P. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.010 · Cited by 114 · cited 1× · see citing works
“encode and store) spatial experiences, but also—and more generally—chains of associated events and sequences of motor actions [77–79].”
BackgroundLearning in and from Brain-Based Devices
Edelman, G. M. (2007) · Science · doi:10.1126/science.1148677 · Cited by 111 · see citing works
A real-world rational agent: unifying old and new AI
Verschure, P. F. M. J., & Althaus, P. (2003) · Cognitive Science · doi:10.1207/s15516709cog2704_1 · Cited by 111 · see citing works
Impaired hippocampal rate coding after lesions of the lateral entorhinal cortex
Lu, L., Leutgeb, J. K., Tsao, A., Henriksen, E. J., Leutgeb, S., Barnes, C. A., Witter, M. P., Moser, M., & Moser, E. I. (2013) · Nature Neuroscience · doi:10.1038/nn.3462 · Cited by 108 · see citing works
Distributed Adaptive Control: A theory of the Mind, Brain, Body Nexus
Verschure, P. F. M. J. (2012) · Biologically Inspired Cognitive Architectures · doi:10.1016/j.bica.2012.04.005 · Cited by 106 · see citing works
The Road Not Taken: Neural Correlates of Decision Making in Orbitofrontal Cortex
Steiner, A. P., & Redish, A. D. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00131 · Cited by 99 · cited 1× · see citing works
“During hippocampal forward sweeps, ventral striatal neurons code covert reward expectations, while orbitofrontal neurons code expected outcome at alternative sites that had not been selected for visiting [129,130].”
BackgroundPezzulo, G., & Castelfranchi, C. (2007) · Cognitive Processing · doi:10.1007/s10339-007-0164-0 · Cited by 97 · cited 1× · see citing works
“However, through the state space learning systems of the AL and the task learning systems of the CL, these goals become incrementally more abstracted from their homeostatic origins (see also [137,138]).”
BackgroundRhodes, S. E. V., & Murray, E. A. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.4374-12.2013 · Cited by 96 · see citing works
Marcos, E., Pani, P., Brunamonti, E., Deco, G., Ferraina, S., & Verschure, P. F. M. J. (2013) · Neuron · doi:10.1016/j.neuron.2013.02.006 · Cited by 92 · cited 1× · see citing works
“…during a countermanding task has shown that the inter trial variability of the neuronal response to the movement cue is directly proportional to the errors the animal has committed (i.e. task memory) and fully predicts performance, i.e. error rate and reaction time rather than firing rate [114].”
BackgroundGoltstein, P. M., Coffey, E. B. J., Roelfsema, P. R., & Pennartz, C. M. A. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.1341-12.2013 · Cited by 88 · see citing works
Sex differences in the neural circuit that mediates female sexual receptivity
Flanagan-Cato, L. (2011) · Frontiers in neuroendocrinology (Print) · doi:10.1016/j.yfrne.2011.02.008 · Cited by 86 · cited 1× · see citing works
“[45–49]).”
Model‐based learning and the contribution of the orbitofrontal cortex to the model‐free world
McDannald, M. A., Takahashi, Y. K., Lopatina, N., Pietras, B. W., Jones, J. L., & Schoenbaum, G. M. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07982.x · Cited by 84 · see citing works
Mulder, A. B., Nordquist, R. E., Örgüt, O., & Pennartz, C. M. A. (2003) · Behavioural Brain Research · doi:10.1016/j.bbr.2003.09.016 · Cited by 84 · cited 1× · see citing works
“Neurophysiological studies in rodents and primates indicate that prefrontal neurons can encode task rules that need to be followed to obtain a goal [94], individual actions or chunks of actions leading up to a goal [95–97], and goals and goal sites themselves [98,99].”
BackgroundThe orbitofrontal cortex, predicted value and choice
Balleine, B. W., Leung, B. K., & Ostlund, S. B. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06270.x · Cited by 80 · see citing works
The Mechanism of Rate Remapping in the Dentate Gyrus
Rennó‐Costa, C., Lisman, J. E., & Verschure, P. F. M. J. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.024 · Cited by 80 · cited 1× · see citing works
“However, the inputs to the hippocampus from the grid cells of the medial entorhinal cortex can be interpreted as representing heading direction and thus action, a source of information that in turn strongly dominates memory dynamics [72].”
BackgroundCorbit, L. H., Leung, B. K., & Balleine, B. W. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3271-13.2013 · Cited by 75 · cited 1× · see citing works
“…context, the amygdaloid complex should be included as part of a larger network for affectively driven goal-directed behaviour not only mediating value based Pavlovian response behaviours, but should also be goal-oriented instrumental behaviours which can be invigorated by Pavlovian cues [111,112].”
BackgroundLisman, J. E. (2007) · Progress in brain research · doi:10.1016/s0079-6123(07)63033-7 · Cited by 67 · see citing works
Memory reactivation and consolidation during sleep: from cellular mechanisms to human performance
Pennartz, C. M. A., Uylings, H. B. M., Barnes, C. A., & McNaughton, B. L. (2002) · Progress in brain research · doi:10.1016/s0079-6123(02)38076-2 · Cited by 59 · cited 2× · see citing works
“These hippocampal replay events are likely to impact on the ventral striatum, where replay events have been shown to occur in succession to place-cell replay in the hippocampus [85,87].”
BackgroundMicrostimulation of Frontal Cortex Can Reorder a Remembered Spatial Sequence
Histed, M. H., & Miller, E. K. (2006) · PLoS Biology · doi:10.1371/journal.pbio.0040134 · Cited by 59 · cited 1× · see citing works
“Neurophysiological studies in rodents and primates indicate that prefrontal neurons can encode task rules that need to be followed to obtain a goal [94], individual actions or chunks of actions leading up to a goal [95–97], and goals and goal sites themselves [98,99].”
BackgroundOrganization and physiology of thesubstantia nigra
Condé, H. (1992) · Experimental Brain Research · doi:10.1007/bf02259099 · Cited by 57 · see citing works
A Signature of Attractor Dynamics in the CA3 Region of the Hippocampus
Rennó‐Costa, C., Lisman, J. E., & Verschure, P. F. M. J. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003641 · Cited by 52 · see citing works
Place fields of rat hippocampal pyramidal cells and spatial learning in the watermaze
Hollup, S., Molden, S., Donnett, J. G., Moser, M., & Moser, E. I. (2001) · European Journal of Neuroscience · doi:10.1046/j.0953-816x.2001.01487.x · Cited by 49 · see citing works
Herreros, I., & Verschure, P. F. M. J. (2013) · Neural Networks · doi:10.1016/j.neunet.2013.01.026 · Cited by 49 · see citing works
Transient Neuronal Correlations Underlying Goal Selection and Maintenance in Prefrontal Cortex
Tsujimoto, S., Genovesio, A., & Wise, S. P. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhn033 · Cited by 49 · cited 1× · see citing works
“Neurophysiological studies in rodents and primates indicate that prefrontal neurons can encode task rules that need to be followed to obtain a goal [94], individual actions or chunks of actions leading up to a goal [95–97], and goals and goal sites themselves [98,99].”
BackgroundMathews, Z., Lechón, M., Calvo, J. M. B., Dhir, A., Duff, A., Badia, S. B. I., & Verschure, P. F. M. J. (2009) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2009.5354264 · Cited by 47 · see citing works
Lansink, C. S., Jackson, J. C., Lankelma, J. V., Ito, R., Robbins, T. W., Everitt, B. J., & Pennartz, C. M. A. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.0593-12.2012 · Cited by 42 · see citing works
A Mechanistic Approach to Preventing Schizophrenia in At-Risk Individuals
Moghaddam, B. (2013) · Neuron · doi:10.1016/j.neuron.2013.03.024 · Cited by 38 · see citing works
Verschure, P. F. M. J., & Pfeifer, R. (1993) · The MIT Press eBooks · doi:10.7551/mitpress/3116.003.0030 · Cited by 36 · see citing works
Duff, A., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2010) · Brain Research Bulletin · doi:10.1016/j.brainresbull.2010.11.008 · Cited by 34 · see citing works
ALLOSTATIC CONTROL FOR ROBOT BEHAVIOR REGULATION: A COMPARATIVE RODENT-ROBOT STUDY
Sánchez-Fibla, M., Bernardet, U., Wasserman, E., Pelc, T., Mintz, M., Jackson, J. C., Lansink, C. S., Pennartz, C. M. A., & Verschure, P. F. M. J. (2010) · Advances in Complex Systems · doi:10.1142/s0219525910002621 · Cited by 32 · cited 3× · see citing works
“In addition, these functions address not only homeostatic regulation, but also allostasis, referring especially to responses to challenges that require system-wide, dynamic adaptation and predictive regulation in anticipation of upcoming homeostatic disturbances [17,18].”
Backgroundvan Wingerden, M., Vinck, M. A., Tijms, V., Ferreira, I. R. S., Jonker, A. J., & Pennartz, C. M. A. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.039 · Cited by 31 · cited 2× · see citing works
“Thus, especially the orbitofrontal cortex may not only provide model-based information on cue/object value before choices are made [102,103,131], but also engage in post-decisional evaluation and ‘looking back’ on previous decisions.”
BackgroundAllostatic control for robot behaviour regulation: An extension to path planning
Sánchez-Fibla, M., Bernardet, U., & Verschure, P. F. M. J. (2010) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2010.5652866 · Cited by 29 · see citing works
Modulating Behaviors Using Allostatic Control
Vouloutsi, V., Lallée, S., & Verschure, P. F. M. J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39802-5_25 · Cited by 25 · cited 1× · see citing works
“This idea has its roots in Gestalt psychology [20] and is a standard approach towards robot navigation [21] but generalizes towards optimal path planning in mobile robots [22] and social interaction in humanoid robots [23].”
BackgroundUnifying perceptual and behavioral learning with a correlative subspace learning rule
Duff, A., & Verschure, P. F. M. J. (2010) · Neurocomputing · doi:10.1016/j.neucom.2009.11.048 · Cited by 17 · cited 3× · see citing works
“The CL of DAC models how state-action sequences that lead to reinforcement can be stored and increasingly support more complex goal-directed strategies [25,27].”
BackgroundSánchez-Fibla, M., Duff, A., & Verschure, P. F. M. J. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6094939 · Cited by 15 · see citing works
An integrated computational model of the two phase theory of classical conditioning
Inderbitzin, M., Herreros-Alonso, I., & Verschure, P. (2010) · IEEE International Joint Conference on Neural Network · doi:10.1109/ijcnn.2010.5596874 · Cited by 11 · cited 2× · see citing works
“The CL of DAC models how state-action sequences that lead to reinforcement can be stored and increasingly support more complex goal-directed strategies [25,27].”
BackgroundMathews, Z., Verschure, P. F. M. J., & Badia, S. B. I. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509935 · Cited by 10 · cited 1× · see citing works
“This transition towards allocentric goal-oriented navigation leads to robust robot multi-modal (proximity sensors, chemosensing and vision) foraging including recovery from kidnapping and noise [35] and optimal maze navigation in the presence of distracters [36].”
BackgroundThe Hierarchical Accumulation of Knowledge in the Distributed Adaptive Control Architecture
Marcos, E., Ringwald, M., Duff, A., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2013) · Springer eBooks · doi:10.1007/978-3-642-39875-9_10 · Cited by 8 · see citing works
TERANISHI, R. (1989) · The journal of the INCE of Japan · doi:10.11372/souonseigyo1977.13.243 · Cited by 6 · see citing works
Marcos, E., Duff, A., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2010) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596919 · Cited by 5 · see citing works
(2012) · The MIT Press eBooks · doi:10.7551/mitpress/9529.003.0014 · Cited by 5 · cited 1× · see citing works
“It is still largely unknown how such a drive for novel information or exploration (or reduction of uncertainty [52–55]) would be realized by the brain, although some higher-level control systems have been implicated (e.g. prefrontal cortex and hippocampus; see below).”
BackgroundThe Complementary Roles of Allostatic and Contextual Control Systems in Foraging Tasks
Marcos, E., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15193-4_35 · Cited by 4 · see citing works
Ringwald, M., & Verschure, P. F. M. J. (2007) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-93221 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 8
- Verschure PFMJ, 2013, Formal minds and biological brains II: from the mirage of intelligence to a science and engineering of consciousness, IEEE Expert., 28, 33
- Panksepp J, 2012, The archaeology of mind: neuroevolutionary origins of human emotions (Norton series on interpersonal neurobiology)
- Sanchez-Fibla MS, 2010, IEEE/RSJ Int. Conf. 18–22 October 2010
- Miller S, 1928, Sur une forme particuliere des reflexes conditionnels, C.R. Seances Soc. Polonaise Biol., 49, 1155
- Ringwald M, 2007, ECMR 2007–3rd European Conf. on Mobile Robots
- Simerly R, 1995, The rat nervous system, 353
- Gray JA, 2000, The neuropsychology of anxiety: an enquiry into thé functions of thesepto-hippocampal system
- Verschure PFMJ, 2012, The distributed adaptive control architecture of the mind, brain, body nexus, Biol. Inspired Cogn. Archit., 1, 55
Cited by
Sorted by citations
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., O⿿Doherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674 · see citing works
Active Inference, homeostatic regulation and adaptive behavioural control
Pezzulo, G., Rigoli, F., & Friston, K. (2015) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2015.09.001 · Cited by 650 · see citing works
“Some of these architectures also include a (simplified) internal physiology that grounds higher-level goals (Verschure et al., 2014).”
InfluentialCaligiore, D., Pezzulo, G., Baldassarre, G., Bostan, A. C., Strick, P. L., Doya, K., Helmich, R. C. G., Dirkx, M. F., Houk, J. C., Jörntell, H., et al. (2016) · The Cerebellum · doi:10.1007/s12311-016-0763-3 · Cited by 423 · see citing works
Navigating the Affordance Landscape: Feedback Control as a Process Model of Behavior and Cognition
Pezzulo, G., & Cisek, P. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.03.013 · Cited by 419 · see citing works
Hierarchical active inference: A theory of motivated control
Pezzulo, G., Rigoli, F., & Friston, K. J. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.01.009 · Cited by 334 · see citing works
“Calling on early cybernetic models [16–18], the view that the brain uses control hierarchies has inspired many recent proposals [19–25]; for example, hierarchical temporal structures [26], hierarchical reinforcement learning [9], hierarchical mixture of experts [23], distributed adaptive control [8,27] and hierarchica…”
Self-citationLevin, M. G., Pezzulo, G., & Finkelstein, J. M. (2017) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-071114-040647 · Cited by 287 · see citing works
“, from biophysically realistic neural networks to more abstract models; from single-neuron models to large neural networks) have often been useful, as they can provide detailed process models of neural computations and produce quantitative and empirically testable predictions (186).”
Pezzulo, G., & Levin, M. G. (2016) · Journal of The Royal Society Interface · doi:10.1098/rsif.2016.0555 · Cited by 231 · see citing works
Embodied choice: how action influences perceptual decision making
Lepora, N. F., & Pezzulo, G. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004110 · Cited by 209 · see citing works
“We make the case that as cognitive processes are studied in ever more ecological contexts, the importance of the cognitive embodiment will become increasingly apparent [65–67].”
BackgroundSelf-citationShin, H., Son, Y., Chae, U., Kim, J., Choi, N., Lee, H. J., Woo, J., Cho, Y., Yang, S. H., Lee, C. J., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-11628-5 · Cited by 169 · see citing works
Pezzulo, G., & Levin, M. G. (2015) · Integrative Biology · doi:10.1039/c5ib00221d · Cited by 167 · see citing works
Decision-Making From the Animal Perspective: Bridging Ecology and Subjective Cognition
Budaev, S. V., Jørgensen, C., Mangel, M., Eliassen, S., & Giske, J. (2019) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2019.00164 · Cited by 116 · see citing works
The evolution of brain architectures for predictive coding and active inference
Pezzulo, G., Parr, T., & Friston, K. J. (2021) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2020.0531 · Cited by 126 · see citing works
“Rather, older solutions largely remain available and can be selected in the appropriate conditions—thus rendering action selection context-sensitive [5,63].”
BackgroundCorcoran, A. W., Pezzulo, G., & Hohwy, J. (2020) · Biology & Philosophy · doi:10.1007/s10539-020-09746-2 · Cited by 161 · see citing works
Agnès Lacreuse, A., Raz, N., Schmidtke, D., Hopkins, W. D., & Herndon, J. G. (2020) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2019.0618 · Cited by 101 · see citing works
“Living organisms adapt to their environment through processing information [1,2].”
BackgroundBiomaterials via Peptide Assembly: Design, Characterization, and Application in Tissue Engineering
Gray, V. P., Amelung, C. D., Duti, I. J., Laudermilch, E. G., Letteri, R. A., & Lampe, K. J. (2021) · Acta Biomaterialia · doi:10.1016/j.actbio.2021.10.030 · Cited by 90 · see citing works
Active inference and robot control: a case study
Pio‐Lopez, L., Nizard, A., Friston, K. J., & Pezzulo, G. (2016) · Journal of The Royal Society Interface · doi:10.1098/rsif.2016.0616 · Cited by 84 · see citing works
How deep is the brain? The shallow brain hypothesis
Suzuki, M., Pennartz, C. M. A., & Aru, J. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00756-z · Cited by 79 · see citing works
Self-citationEvaluating the Brain Disease Model of Addiction
Heather, N., Field, M., Moss, A. C., & Satel, S. L. (2022) · Routledge eBooks · doi:10.4324/9781003032762 · Cited by 80 · see citing works
Moulin-Frier, C., Fischer, T., Petit, M., Pointeau, G., Puigbò, J.-Y., Pattacini, U., Low, S. C., Camilleri, D., Nguyen, P. D. H., Hoffmann, M., et al. (2017) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2017.2754143 · Cited by 77 · see citing works
“In this paper, we adopt the principles of the Distributed Adaptive Control theory of the mind and the brain (DAC, [39], [40]).”
Developmental perspectives on risky and impulsive choice
Rosenbaum, G. M., & Hartley, C. A. (2018) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2018.0133 · Cited by 68 · see citing works
Rusu, S. I., & Pennartz, C. M. A. (2019) · Hippocampus · doi:10.1002/hipo.23167 · Cited by 63 · see citing works
“A key proposition holds that the HPC does not encode a “task space” as proposed for PFC, but a representation of world states (including the subject's body state; cf. Verschure et al., 2014).”
BackgroundGoal-directed actions transiently depend on dorsal hippocampus
Bradfield, L. A., Leung, B. K., Boldt, S., Liang, S., & Balleine, B. W. (2020) · Nature Neuroscience · doi:10.1038/s41593-020-0693-8 · Cited by 65 · see citing works
Maffei, G., Santos-Pata, D., Marcos, E., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.10.004 · Cited by 58 · see citing works
Donnarumma, F., Maisto, D., & Pezzulo, G. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004864 · Cited by 58 · see citing works
Maisto, D., Donnarumma, F., & Pezzulo, G. (2015) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.1335 · Cited by 59 · see citing works
Functional Oxides for Photoneuromorphic Engineering: Toward a Solar Brain
Pérez‐Tomás, A. (2019) · Advanced Materials Interfaces · doi:10.1002/admi.201900471 · Cited by 50 · see citing works
Treating Brain Disorders by Targeting Adult Neural Stem Cells
Bao, H., & Song, J. (2018) · Trends in Molecular Medicine · doi:10.1016/j.molmed.2018.10.001 · Cited by 51 · see citing works
Synthetic consciousness: the distributed adaptive control perspective
Verschure, P. F. M. J. (2016) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2015.0448 · Cited by 51 · see citing works
Model-Based Approaches to Active Perception and Control
Pezzulo, G., Donnarumma, F., Iodice, P., Maisto, D., & Stoianov, I. P. (2017) · Entropy · doi:10.3390/e19060266 · Cited by 41 · see citing works
A thalamocortical circuit for updating action-outcome associations
Fresno, V., Parkes, S. L., Faugère, A., Coutureau, É., & Wolff, M. (2019) · eLife · doi:10.7554/elife.46187 · Cited by 45 · see citing works
“However, mounting experimental evidence indicates that functional interactions between prefrontal and subcortical areas are essential to support these functions (Balleine and O’Doherty, 2010; Verschure et al., 2014).”
BackgroundZhang, S., Wang, W., Zhornitsky, S., & Li, C.-S. R. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00344 · Cited by 40 · see citing works
Lateral parietal cortex in the generation of behavior: Implications for apathy
Tumati, S., Martens, S., de Jong, B. M., & Aleman, A. (2018) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2018.12.003 · Cited by 40 · see citing works
“the goal and the reason for its choice; the ‘what’ component represents the broad plan of action; the ‘when’ and ‘where’ components reflect the spatiotemporal aspects of the actions; the ‘how’ component describes the detailed motor sequences to be performed (Verschure et al., 2014).”
Prefrontal Goal Codes Emerge as Latent States in Probabilistic Value Learning
Stoianov, I., Genovesio, A., & Pezzulo, G. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00886 · Cited by 44 · see citing works
“…Solway and Botvinick, 2012; Pezzulo et al., 2013) and active inference (Friston, 2010; Clark, 2013), where they have a key role in guiding action selection and control (Lepora and Pezzulo, 2015, Pezzulo and Castelfranchi, 2009, Pezzulo et al., 2014a, Pezzulo et al., 2014b, Verschure et al., 2014).”
BackgroundA Liaison Brought to Light: Cerebellum-Hippocampus, Partners for Spatial Cognition
Rondi‐Reig, L., Paradis, A., & Fallahnezhad, M. (2022) · The Cerebellum · doi:10.1007/s12311-022-01422-3 · Cited by 39 · see citing works
“However, the fact that these were not correlated with possible motor deficits confirmed the cerebellum as a crucial brain region for goal-directed navigation, independently of its motor role ([21–23] but also see [24]).”
BackgroundThe Synthetic Psychology of the Self
Prescott, T. J., & Camilleri, D. (2018) · International series on intelligent systems, control and automation: science and engineering · doi:10.1007/978-3-319-97550-4_7 · Cited by 38 · see citing works
“For the past seven years, together with European colleagues, we have also been incorporating models of key brain systems into a brain-inspired control architecture for the iCub robot (Figure 1) called distributed adaptive control (DAC), developed by Paul Verschure and colleagues [69-71].”
BackgroundModel-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysis
Stoianov, I. P., Pennartz, C. M. A., Lansink, C. S., & Pezzulo, G. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006316 · Cited by 37 · see citing works
“This reliance on a relatively low-dimensional state space makes spatial navigation quite special and tractable, both in machine learning and the brain; it is thus possible that brain areas that support goal-directed processing outside navigation domains rely on more complex computational solutions that remain to be fu…”
BackgroundReadiness to remember: predicting variability in episodic memory
Madore, K. P., & Wagner, A. D. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.05.006 · Cited by 37 · see citing works
Renewal of goal direction with a context change after habit learning.
Steinfeld, M. R., & Bouton, M. E. (2021) · Behavioral Neuroscience · doi:10.1037/bne0000422 · Cited by 34 · see citing works
The principles of goal-directed decision-making: from neural mechanisms to computation and robotics
Pezzulo, G., Verschure, P. F. M. J., Balkenius, C., & Pennartz, C. M. A. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0470 · Cited by 35 · see citing works
Memory and mental time travel in humans and social robots
Prescott, T. J., Camilleri, D., Martínez-Hernández, U., Damianou, A., & Lawrence, N. D. (2019) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2018.0025 · Cited by 34 · see citing works
“This is already happening in the DAC architecture where action plans, represented as graphical models, can be accessed based on goal cues extracted from human speech.”
MethodInfluentialLearning and Judgment Can Be Affected by Predisposed Fearfulness in Laying Hens
Haas, E. N. D., Lee, C., & Rodenburg, T. B. (2017) · Frontiers in Veterinary Science · doi:10.3389/fvets.2017.00113 · Cited by 32 · see citing works
The ventral hippocampus is involved in multi‐goal obstacle‐rich spatial navigation
Contreras, M., Pelc, T., Llofriu, M., Weitzenfeld, A., & Fellous, J. (2018) · Hippocampus · doi:10.1002/hipo.22993 · Cited by 33 · see citing works
Socially adaptive cognitive architecture for human-robot collaboration in industrial settings
Freire, I. T., Guerrero-Rosado, O., Amil, A. F., & Verschure, P. F. M. J. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1248646 · Cited by 31 · see citing works
Active inference and cognitive-emotional interactions in the brain
Pezzulo, G., Barca, L., & Friston, K. J. (2015) · Behavioral and Brain Sciences · doi:10.1017/s0140525x14001009 · Cited by 32 · see citing works
Hypothalamic Responses to Cocaine and Food Cues in Individuals with Cocaine Dependence
Zhang, S., Zhornitsky, S., Le, T. M., & Li, C.-S. R. (2019) · The International Journal of Neuropsychopharmacology · doi:10.1093/ijnp/pyz044 · Cited by 30 · see citing works
“The hypothalamus receives heavy dopaminergic projections from the midbrain and represents a key brain region in regulating arousal, food intake, and sexual drive and in supporting reward and affective responses during a wide range of goal-directed behaviors (DiLeone et al., 2003; Watabe-Uchida et al., 2012; Sternson,…”
BackgroundUnderstanding brain functional architecture through robotics
Prescott, T. J., & Wilson, S. P. (2023) · Science Robotics · doi:10.1126/scirobotics.adg6014 · Cited by 25 · see citing works
“Broadly speaking, these controllers are arranged with fast but inflexible controllers at the bottom and slow but flexible ones at the top (4, 7).”
BackgroundCarvalho, F. M., Chaim, K. T., Sanchez, T. A., & de Araújo, D. B. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00268 · Cited by 23 · see citing works
“Efficient interaction between an agent and its surrounding relies heavily on the use of prospective internal models to enhance processing of expected stimuli, and on the ability to update these models to achieve the desired outcome (Verschure et al., 2014).”
BackgroundMACHINE INTELLIGENCE. ESSAYS ON THE THEORY OF MACHINE LEARNING AND ARTIFICIAL INTELLIGENCE
Шумский, С., & Shumskiy, S. (2019) · Publishing Center RIOR eBooks · doi:10.29039/02011-1 · Cited by 21 · see citing works
Nonparametric Problem-Space Clustering: Learning Efficient Codes for Cognitive Control Tasks
Maisto, D., Donnarumma, F., & Pezzulo, G. (2016) · Entropy · doi:10.3390/e18020061 · Cited by 20 · see citing works
“An open problem in this field is how task-relevant representations might be created in the first place, with alternative theories that link to statistical learning in the PFC proper or assign a role for the hippocampus in rapidly creating these clusters (or schemas [54]) and transferring them to cortical structures [1…”
BackgroundThe Morphospace of ConsciousnessPreprint
Arsiwalla, X. D., Moulin-Frier, C., Herreros, I., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2017) · doi:10.31234/osf.io/emhnp · Cited by 20 · see citing works
“In turn, evolution has shaped cognition in such a way so as to support the organism’s basic survival (using wakefulness/arousal) as well as higher-order drives (using awareness) associated to cooperation and competition in a multi-agent environment [80].”
BackgroundRaina, A., Puentes, L., Cagan, J., & McComb, C. (2021) · Journal of Mechanical Design · doi:10.1115/1.4051013 · Cited by 19 · see citing works
“Making decisions that optimize for specific objectives constitutes a critical skill in problem solving [11].”
BackgroundTowards a Synthetic Tutor Assistant: The EASEL Project and its Architecture
Vouloutsi, V., Blancas, M., Zucca, R., Omedas, P., Reidsma, D., Davison, D. P., Charisi, V., Wijnen, F., van der Meij, J., Evers, V., et al. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-42417-0_32 · Cited by 18 · see citing works
Embodied artificial intelligence through distributed adaptive control: An integrated framework
Moulin-Frier, C., Puigbò, J.-Y., Arsiwalla, X. D., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2017) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2017.8329825 · Cited by 18 · see citing works
“For this aim, we adopt the principles of the Distributed Adaptive Control (DAC) theory of the mind and brain [22], [23].”
MethodRomero‐Martínez, Á., Lila, M., Vitoria-Estruch, S., & Moya‐Albiol, L. (2018) · Journal of Interpersonal Violence · doi:10.1177/0886260518814263 · Cited by 18 · see citing works
How Pain Shapes Depression and Anxiety: A Hybrid Self-regulatory/Predictive Mind Perspective
Karoly, P. (2020) · Journal of Clinical Psychology in Medical Settings · doi:10.1007/s10880-019-09693-5 · Cited by 18 · see citing works
Learning and memory processes in behavioural addiction: A systematic review
Ngetich, R. K., Villalba-García, C., Soborun, Y., Vékony, T., Czakó, A., Demetrovics, Z., & Németh, D. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105747 · Cited by 20 · see citing works
de Maele, T. V., Dhoedt, B., Verbelen, T., & Pezzulo, G. (2024) · Nature Communications · doi:10.1038/s41467-024-54257-3 · Cited by 20 · see citing works
How? Why? What? Where? When? Who? Grounding Ontology in the Actions of a Situated Social Agent
Lallée, S., & Verschure, P. F. M. J. (2015) · Robotics · doi:10.3390/robotics4020169 · Cited by 16 · see citing works
Maier, M., Bañuelos, N. L., Ballester, B. R., Duarte, E., & Verschure, P. F. M. J. (2017) · International Conference on Rehabilitation Robotics · doi:10.1109/icorr.2017.8009371 · Cited by 17 · see citing works
“We adopted a finer gradient of impairment level as classically reported [31], [32].”
BackgroundMethodWerlen, E., & Jones, M. W. (2015) · Progress in brain research · doi:10.1016/bs.pbr.2015.03.002 · Cited by 16 · see citing works
Learning to select actions shapes recurrent dynamics in the corticostriatal system
Márton, C. D., Schultz, S. R., & Averbeck, B. B. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.09.008 · Cited by 16 · see citing works
Modeling the formation of social conventions from embodied real-time interactions
Freire, I. T., Moulin-Frier, C., Sánchez-Fibla, M., Arsiwalla, X. D., & Verschure, P. F. M. J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0234434 · Cited by 18 · see citing works
“As discussed in [26, 41], this will facilitate the integration of sensory-motor contingencies into a long-term memory that allows for learning of rules.”
Is Consciousness a Control Process?
Arsiwalla, X. D., Herreros, I., Moulin-Frier, C., Marti, S. B., & Verschure, P. F. M. J. (2016) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-61499-696-5-233 · Cited by 14 · see citing works
Towards Self-controlled Robots Through Distributed Adaptive Control
Puigbò, J.-Y., Moulin-Frier, C., & Verschure, P. F. M. J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-42417-0_52 · Cited by 13 · see citing works
Li, M., Fan, J., Cheng, S., Yang, L., Yang, L., Wang, L., Shang, Z., & Wan, H. (2021) · Animals · doi:10.3390/ani11072003 · Cited by 13 · see citing works
“, ‘What is the route from the current location to the known goal location?’) [2,3].”
BackgroundSyngap1 promotes cognitive function through regulation of cortical sensorimotor dynamics
Vaissière, T., Michaelson, S. D., Creson, T. K., Goins, J., Fürth, D., Balázsfi, D., Rojas, C., Golovin, R. M., Meletis, K., Miller, C. A., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-56125-0 · Cited by 12 · see citing works
“To explore this, we utilized behavioral paradigms that rely on passive (receptive) and active (generative) whisker sensing to drive perceptual learning 43,44 .”
Visual anticipation biases conscious decision making but not bottom-up visual processing
Mathews, Z., Cetnarski, R., & Verschure, P. F. M. J. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01443 · Cited by 12 · see citing works
Autonomous development of turn-taking behaviors in agent populations: A computational study
Moulin-Frier, C., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2015) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2015.7346139 · Cited by 11 · see citing works
“This rather strong hypothesis, which requires that each agent is able to identify a conspecific from the acoustic properties of its calls, is plausible for several social species (see e.g. [13] for the case of adult marmosets).”
BackgroundHatzipantelis, C. J., Langiu, M., Vandekolk, T. H., Pierce, T. L., Nithianantharajah, J., Stewart, G. D., & Langmead, C. J. (2020) · ACS Pharmacology & Translational Science · doi:10.1021/acsptsci.0c00117 · Cited by 12 · see citing works
A global framework for a systemic view of brain modeling
Alexandre, F. (2021) · Brain Informatics · doi:10.1186/s40708-021-00126-4 · Cited by 11 · see citing works
Maranesi, M., Bruni, S., Livi, A., Donnarumma, F., Pezzulo, G., & Bonini, L. (2019) · Scientific Reports · doi:10.1038/s41598-019-48216-y · Cited by 10 · see citing works
Goekoop, R., & de Kleijn, R. (2021) · Entropy · doi:10.3390/e23121701 · Cited by 9 · see citing works
“Another way to refer to such hierarchical predictive models of econiches is a ‘goal hierarchy’ [20,78].”
BackgroundRusu, S., Bos, J., Marchesi, P., Lankelma, J., Pica, I. F., Gentet, L., Joëls, M., & Pennartz, C. (2024) · iScience · doi:10.1016/j.isci.2024.109205 · Cited by 5 · see citing works
“32,100,101 How OFC replay may contribute to the formation of an internal model of task space 22,23,24 is difficult to tell at present.”
BackgroundRepresentation and the active consumer
Butlin, P. (2018) · Synthese · doi:10.1007/s11229-018-01941-9 · Cited by 9 · see citing works
Control systems theory revisited: new insights on the brain clocks of time-to-action
Ferber, S. G., Weller, A., & Soreq, H. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1171765 · Cited by 8 · see citing works
“While the elements within the parallel sequences are standardly ordered when operating within the same timeframe, that timeframe depends on the range of the required signal variability (Verschure et al., 2014; Sendhilnathan et al., 2021) for a given goal-oriented behavior.”
BackgroundThe Morphospace of Consciousness: Three Kinds of Complexity for Minds and Machines
Arsiwalla, X. D., Solé, R. V., Moulin-Frier, C., Herreros, I., Sánchez-Fibla, M., & Verschure, P. F. M. J. (2023) · NeuroSci · doi:10.3390/neurosci4020009 · Cited by 8 · see citing works
“Social interactions play a crucial role in driving and regulating adaptive responses through behavioral feedback in both natural and artificial systems [20].”
BackgroundMoulin-Frier, C., Arsiwalla, X. D., Puigbò, J.-Y., Sánchez-Fibla, M., Duff, A., & Verschure, P. F. M. J. (2016) · neural information processing systems · Cited by 8 · see citing works
About the goal of a goals’ goal theory
Herreros, I., & Verschure, P. F. M. J. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1051952 · Cited by 8 · see citing works
Verbeke, W., & Masih, J. (2020) · Journal of Personal Selling & Sales Management · doi:10.1080/08853134.2019.1654389 · Cited by 6 · see citing works
“The task and action space is known as a set of rules, goals, and goal-predictive values (Verschure, Pennartz, and Pezzulo 2014) that are needed for using tools, solving concrete problems, and making choices.”
BackgroundMethodInfluentialOutline of a cybernetic theory of brain function based on neural timing nets
Cariani, P. (2015) · Kybernetes · doi:10.1108/k-11-2014-0242 · Cited by 7 · see citing works
Modeling the Formation of Social Conventions in Multi-Agent Populations.Preprint
Freire, I. T., Moulin-Frier, C., Sánchez-Fibla, M., Arsiwalla, X. D., & Verschure, P. F. M. J. (2018) · arXiv (Cornell University) · Cited by 7 · see citing works
Epistemic Autonomy: Self-supervised Learning in the Mammalian Hippocampus
Santos-Pata, D., Amil, A. F., Raikov, I., Rennó‐Costa, C., Mura, A., Soltész, I., & Verschure, P. F. M. J. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.03.016 · Cited by 7 · see citing works
“…network of the cortico-hippocampal system is a generic substrate for self-supervised gradient descent learning in the context of the overall brain architecture comprising dynamically coupled perceptual, cognitive, and motor systems [106] that each in turn are organized as epistemically closed.”
BackgroundSequential memory improves sample and memory efficiency in episodic control
Freire, I. T., Amil, A. F., & Verschure, P. F. M. J. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00950-3 · Cited by 6 · see citing works
“Thus far, ERL models do not capitalize on the sequential nature of the temporal unfolding of events that are reflected in the structure of hippocampal memory 21–23 .”
Perkes, I. E., Morris, R. W., Griffiths, K. R., Quail, S. L., Waters, F. A., O’Brien, M., Hazell, P., & Balleine, B. W. (2022) · Biological Psychiatry Global Open Science · doi:10.1016/j.bpsgos.2022.11.004 · Cited by 6 · see citing works
“Nevertheless, the way that these values are integrated with action-outcome retrieval during performance and the neural substrates that support that integrative process are underexplored in humans, as is their causal role in action control (12,13).”
BackgroundDopamine receptors of the rodent fastigial nucleus support skilled reaching for goal-directed action
Caragea, V.-M., Méndez‐Couz, M., & Manahan‐Vaughan, D. (2023) · Brain Structure and Function · doi:10.1007/s00429-023-02685-0 · Cited by 5 · see citing works
“Participation in goal-directed behavior has been ascribed to the cerebellum (Babayan et al. 2017; Verschure et al. 2014) and the deep cerebellar nuclei (DCN) (Callu et al. 2013; Xiao et al. 2018).”
BackgroundSimulated Dopamine Modulation of a Neurorobotic Model of the Basal Ganglia
Prescott, T. J., Montes-González, F., Gurney, K. R., Humphries, M. D., & Redgrave, P. (2024) · Biomimetics · doi:10.3390/biomimetics9030139 · Cited by 4 · see citing works
“With excessively high levels of simulated dopamine, the robot displays hyperactivity and rapid switching between behaviours, symptoms that show some resemblance to hyper-dopaminergic outcomes in animals and humans.”
Toward mechanistic models of action-oriented and detached cognition
Pezzulo, G. (2016) · Behavioral and Brain Sciences · doi:10.1017/s0140525x15001648 · Cited by 5 · see citing works
Machine Morality: From Harm-Avoidance to Human-Robot Cooperation
Freire, I. T., Urikh, D., Arsiwalla, X. D., & Verschure, P. F. M. J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-64313-3_13 · Cited by 5 · see citing works
From motor to visually guided bimanual affordance learning
Sánchez-Fibla, M., Forestier, S., Moulin-Frier, C., Puigbò, J.-Y., & Verschure, P. F. M. J. (2019) · Adaptive Behavior · doi:10.1177/1059712319855836 · Cited by 5 · see citing works
Decoding route selection of pigeon during goal-directed behavior: A joint spike-LFP study
Cheng, S., Li, M., Fan, J., Shang, Z., & Wan, H. (2021) · Behavioural Brain Research · doi:10.1016/j.bbr.2021.113289 · Cited by 5 · see citing works
Esmaeili, A., Antonova, A. M., Sitnikova, E., & Smirnov, K. (2021) · Behavioural Brain Research · doi:10.1016/j.bbr.2021.113627 · Cited by 4 · see citing works
Two possible driving forces supporting the evolution of animal communication
Moulin-Frier, C., & Verschure, P. F. M. J. (2016) · Physics of Life Reviews · doi:10.1016/j.plrev.2016.01.019 · Cited by 4 · see citing works
Fan, M., Liu, Z., Wei, A., Gao, R.-J., Luan, P., & He, X. (2025) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s545022 · Cited by 3 · see citing works
“14 Furthermore, emerging evidence suggests that abnormalities in interoceptive-affective integration may contribute to the development of anomalous self-experiences characteristic of schizophrenia.”
Human foraging strategies flexibly adapt to resource distribution and time constraints
Simonelli, V., Nuzzi, D., Lancia, G. L., & Pezzulo, G. (2025) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-025-01347-4 · Cited by 3 · see citing works
“…that arise during foraging and related tasks have been widely studied in humans and other animals (Cohen et al., 2007; Mehlhorn et al., 2015; Addicott et al., 2017; Daw et al., 2006; Verschure et al., 2014; Maffei et al., 2015; Parr et al., 2022; Pezzulo et al., 2018; Friston et al., 2017).”
Shinya, Y., Oku, K., Watanabe, H., Taga, G., & Fujii, S. (2022) · Experimental Brain Research · doi:10.1007/s00221-022-06379-7 · Cited by 3 · see citing works
“In turn, the physiological needs may enhance auditory-motor learning and goal-directed behavior in young infants (Ecker and Gilead 2018; Nagai 2019; Verschure et al. 2014).”
BackgroundNikrahan, G. R. (2023) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2023.1050164 · Cited by 3 · see citing works
“Xu et al. (2012) found that the quality and stability of marriage until the 40th month of the relationship is predicted by lower activation of medial OFC, right accumbens, and right subcallosal cingulate during the early stages of love (Xu et al., 2012).”
BackgroundInfluentialThe Role of Hp-NCL Network in Goal-Directed Routing Information Encoding of Bird: A Review
Li, M., Shang, Z., Zhao, K., Cheng, S., & Wan, H. (2020) · Brain Sciences · doi:10.3390/brainsci10090617 · Cited by 3 · see citing works
“In this complex behavior, the brain mechanisms of the basic functions of environmental perception and behavioral decision have been widely studied [3].”
BackgroundEffenberg, A. O., & Schmitz, G. H. (2022) · Springer eBooks · doi:10.1007/978-3-662-53386-4_60-1 · Cited by 2 · see citing works
A Model of Agential Learning Using Active Inference
Pitliya, R. J., & Murphy, R. A. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-47958-8_8 · Cited by 2 · see citing works
Neural activity in monkey amygdala during performance of a multisensory operant task
Montes-Lourido, P., Vicente, A. F., Bermúdez, M. A., & González, F. (2015) · Journal of Integrative Neuroscience · doi:10.1142/s021963521550020x · Cited by 2 · see citing works
- Authors
- Verschure, P. F. M. J., Pennartz, C. M. A., & Pezzulo, G.
- Affiliations
- Institució Catalana de Recerca i Estudis Avançats (ES)
- University of Amsterdam (NL)
- National Research Council (IT)
- Corresponding author
- Verschure, Paul F. M. J.
- Venue
- Philosophical Transactions of the Royal Society B Biological Sciences
- Publisher
- The Royal Society
- Published
- 2014-11-05
- Volume
- 369
- Issue
- 1655
- Pages
- 20130483
- ISSN
- 0962-8436 (ISSN-L)
- 1471-2970
- PMC
- PMC4186236
- OpenAlex
- W2143590399
- Semantic Scholar
- defe4585fd20817ba121287456fe83112d9582f7
- Cited by
- 131
- References
- 152
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 11 · 2023: 11 · 2022: 9 · 2021: 16 · 2020: 11 · 2019: 13 · 2018: 11 · 2017: 10 · 2016: 15 · 2015: 16 · 2014: 1
- FWCI
- 1.35
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Philosophical Transactions of the Royal Society B Biological Sciences (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion)
- Repositori digital de la UPF (Universitat Pompeu Fabra) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Keywords
- goal-directed behavior, distributed adaptive control, embodied cognition, outcome prediction, valence, basal ganglia, hippocampus, prefrontal cortex, amygdala, hypothalamus, action planning, learning and memory, neurorobotics
- MeSH terms
- Animals, Artificial Intelligence, Behavior, Animal, Brain, Choice Behavior, Cognition, Goals, Rodentia
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
