The DOI matches its Crossref record; 135 of the 136 references it cites were found in a registry or an index; none of the 99 sources its claims cite is checked for support yet.
Habits without values.
Open accessPublished version of a preprint
Abstract
Habits form a crucial component of behavior. In recent years, key computational models have conceptualized habits as arising from model-free reinforcement learning mechanisms, which typically select between available actions based on the future value expected to result from each. Traditionally, however, habits have been understood as behaviors that can be triggered directly by a stimulus, without requiring the animal to evaluate expected outcomes. Here, we develop a computational model instantiating this traditional view, in which habits develop through the direct strengthening of recently taken actions rather than through the encoding of outcomes. We demonstrate that this model accounts for key behavioral manifestations of habits, including insensitivity to outcome devaluation and contingency degradation, as well as the effects of reinforcement schedule on the rate of habit formation. The model also explains the prevalent observation of perseveration in repeated-choice tasks as an additional behavioral manifestation of the habit system. We suggest that mapping habitual behaviors onto value-free mechanisms provides a parsimonious account of existing behavioral and neural data. This mapping may provide a new foundation for building robust and comprehensive models of the interaction of habits with other, more goal-directed types of behaviors and help to better guide research into the neural mechanisms underlying control of instrumental behavior more generally. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper develops a computational model in which habits are value-free stimulus-response associations strengthened by repetition, rather than model-free reinforcement learning. It shows this model reproduces key habit phenomena, including insensitivity to outcome devaluation and contingency degradation, reinforcement-schedule effects on habit formation, lesion-like dissociations, and perseveration.
Key findings
- A value-free habitual controller, combined with a model-based goal-directed controller and an arbiter, reproduces overtraining-induced inflexibility: after longer training, simulated agents continued lever-pressing under omission and devaluation, whereas shorter training allowed reversal. Training duration was varied from 100 to 2000 trials in intervals of 100 for omission and from 10 to 2000 trials in intervals of 100 for devaluation.
- Simulated agents trained on variable-interval (VI6: pellets baited every 6 s, on average 10/min) lever-pressed at a moderate rate and habitized early, whereas agents trained on variable-ratio (VR10: 10% chance per press) lever-pressed at a high rate and habitized much later.
- Simulated lesions reproduced rodent striatal patterns: decreasing goal-directed parameters (DMS-like) produced a low press rate unaffected by omission or devaluation; decreasing habitual parameters (DLS-like) produced a high press rate that adapted to both manipulations. Control agents adapted after limited training (5,000 trials) but not after extensive training (15,000 trials).
- In a dynamic two-armed bandit task (10,000 trials; 50 model copies; reinforcer probabilities changed by independent Gaussian random walks, SD = 0.15), the model reproduced both reinforcement-seeking and value-free perseveration; replacing the habitual component with model-free RL failed to reproduce the perseverative pattern.
Methods
- Computational model with three modules: a habitual controller tracking recency-weighted action history \(H\), a goal-directed controller using model-based RL to learn separate reinforcer expectations \(R\) and utilities \(U\), and an arbiter weighting control by action-outcome contingency \(g\) and overall habitization \(h\).
- Simulated task environments included reversal learning, omission contingency, outcome devaluation, free-operant VR/VI schedules, simulated lesions, and a two-armed bandit. Most simulations used a single state with two actions; free-operant simulations used continuous press rates from 0 to 150 presses/min with function approximation and an effort cost quadratic in press rate.
- Analyses averaged simulated performance across agents and, for the two-armed bandit, used logistic regression quantifying influences of past actions, reinforcers, and their interaction on future choice.
Limitations
- The paper states that the model is described at an abstract computational level, limiting engagement with neurobiological detail, and that a neurobiologically detailed implementation is a critical next step.
- The paper states that the arbiter’s conditions require future research, that the model is agnostic about whether bypassing model-free RL entirely is appropriate, and that the form of value/motivational modulation of habit learning remains unresolved.
- Evident from the text: the findings come from simulations rather than new empirical experiments, and many simulations use simplified single-state, two-action environments with specific parameter and utility choices, so generality across richer environments is not established.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 99 cited sources checked · 0 supported · 0 flagged
None of the sources cited for this claim have been checked yet.
In the paper
“as well as the effects of reinforcement schedule on the rate of habit formation.”
§ Habits without Values > Abstract
What the sources say
- Not checked yetcites Adams, 1982
- Not checked yetcites Dickinson, 1985
- Not checked yetcites Gremel & Costa, 2013
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We demonstrate that this model accounts for key behavioral manifestations of habits, including insensitivity to outcome devaluation and contingency degradation”
§ Habits without Values > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The model also explains the prevalent observation of perseveration in repeated-choice tasks as an additional behavioral manifestation of the habit system.”
§ Habits without Values > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
None of the sources cited for this claim have been checked yet.
In the paper
“This value-free habit mechanism accounts for key findings in the animal learning literature that dissociate habitual and goal-directed actions, namely the tendency for an animal to continue performing a previously learned action when that action is no longer predictive of the reinforcing outcome (contingency degradation; Hammond, 1980) or when the predicted outcome ceases to be desired by the subj”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Hammond, 1980
- Not checked yetcites Adams & Dickinson, 1981
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As a result, when the reversal occurs, the agent continues to select Action A for an extended period, past the point where the goal-directed controller has learned that Action B is more likely to be reinforced (compare Figures 3B and 3E).”
§ Habits without Values > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“In addition, this model provides what is, to our knowledge, the first computational account of the difference in rate of habit formation under variable-interval and variable-ratio schedules of reinforcement (Adams, 1982; Dickinson, 1985; Gremel & Costa, 2013).”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Adams, 1982
- Not checked yetcites Dickinson, 1985
- Not checked yetcites Gremel & Costa, 2013
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Initially, behavior is driven by the goal-directed controller, which gradually learns the relative reinforcement rates (Figure 3B) and thus increasingly selects Action A.”
§ Habits without Values > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In the following sections we demonstrate that this model can capture all of the key diagnostic features of habitual behavior previously identified in animal behavior, including sensitivity to repetition frequency, reinforcement schedule, and selective modulation by lesions to one of two dissociable neural circuits.”
§ Habits without Values > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition to demonstrating the different kinds of learning that drive each of these controllers, this example demonstrates that our model captures the observation that behavioral control in a novel environment tends to evolve from goal-directed to habitual (i.e., habits form from actions that were originally selected in a value-based manner).”
§ Habits without Values > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A “control” agent, with intact habitual and goal-directed controllers, adopted a high press rate that adapted to both manipulations when given limited training, but did not adapt to either following extensive training.”
§ Habits without Values > Results > Effects of Striatal Lesions on Habit Formation and Behavioral Flexibility
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 21
- Probably not a substantial claim. 18
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“whereas agents trained on VR schedules lever-pressed at a high rate and habitized much later in training (Figure 4, right).”
§ Habits without Values > Results > Effects of Reinforcement Schedule on Habit Formation
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we found that simulated agents trained on VI schedules lever-pressed at a moderate rate and habitized early in training (Figure 4, left)”
§ Habits without Values > Results > Effects of Reinforcement Schedule on Habit Formation
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Model-free algorithms typically operate by learning the expected future reward associated with each possible action or state, relying crucially on these value representations.”
§ Habits without Values > Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Goal-directed behaviors are understood to be driven by consideration of the outcomes that they are likely to bring about (i.e. “action-outcome” representations).”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Balleine & O’Doherty, 2010
- Not checked yetcites Dickinson, 1985
- Not checked yetcites Dolan & Dayan, 2013
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“This category of value-free habits has been maintained in modern theorizing on habits, where it has been referred to as “direct cuing” of behavior, as distinct from value-based forms of habits (“motivated cueing”; Wood & Neal, 2007; Wood & Rünger, 2016).”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Wood & Neal, 2007
- Not checked yetcites Wood & Rünger, 2016
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Modern reinforcement learning (RL) has elaborated this law into a set of computational algorithms, according to which actions are selected based on cached values learned from previous experience (Daw, Niv, & Dayan, 2005; Dolan & Dayan, 2013; Sutton & Barto, 1998).”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Daw, Niv, & Dayan, 2005
- Not checked yetcites Dolan & Dayan, 2013
- Not checked yetcites Sutton & Barto, 1998
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This class of computations focuses only on potential rewards, ignoring all reward-unrelated elements of one’s environment (discussed below); it is therefore referred to as “model-free” RL.”
§ Habits without Values > Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“This formulation for habits has become so prevalent that the terms “habit” and “model-free” are now used interchangeably in much of the computational literature (Dolan & Dayan, 2013; Doll, Simon, & Daw, 2012).”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Dolan & Dayan, 2013
- Not checked yetcites Doll, Simon, & Daw, 2012
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
In the paper
“This law holds that an action that has been taken often in the past is likely to be repeated in the future, independent of its past consequences; in other words, it describes habits as a form of perseveration.”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Thorndike, 1911
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“A large and growing body of work indicates that these behaviors depend on different sets of computations and distinct underlying neural circuits, suggesting that separable goal-directed and habitual systems implement fundamentally different strategies for the control of behavior”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Balleine & O’Doherty, 2010
- Not checked yetcites Dickinson, 1985
- Not checked yetcites Dolan & Dayan, 2013
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Here, we have shown computationally that such value-based mechanisms are not strictly necessary, and that a value-free mechanism can account for the major behavioral phenomena that define habits.”
§ Habits without Values > Discussion
Load-bearingStrength:boosted· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The idea that separate goal-directed and habitual controllers exist in the brain, supported by distinct neural circuits, is strongly supported by lesion data from both humans and other animals.”
§ Habits without Values > Discussion > Tensions in Neuroscientific Data
Strength:boosted· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The finding that VI schedules result in rapid habit formation has been widely replicated and represents a key element of the experimental toolkit for the study of habits (Gremel & Costa, 2013; Tanaka, Balleine, & O’Doherty, 2008; Yin & Knowlton, 2006).”
§ Habits without Values > Results > Effects of Reinforcement Schedule on Habit Formation
What the sources say
- Not checked yetcites Gremel & Costa, 2013
- Not checked yetcites Tanaka, Balleine, & O’Doherty, 2008
- Not checked yetcites Yin & Knowlton, 2006
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The first of these features is that habits form slowly over time and often depend on behaviors that are initially taken under goal-directed control.”
§ Habits without Values > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“We first test whether our model can capture a central finding from research on habits: that extensive training in a static environment (overtraining) can render behavior inflexible in the face of changes to that environment.”
§ Habits without Values > Results > Effects of Training Duration on Behavioral Flexibility
What the sources say
- Not checked yetcites Dickinson, 1998
- Not checked yetcites Adams, 1982
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This example not only begins to validate the predictive abilities of our particular model, but also highlights the importance of a value-free habitual controller more generally in explaining habit-like behaviors that cannot otherwise be accounted for by a model-free RL-based algorithm alone.”
§ Habits without Values > Results > Perseverative Behavior in Sequential Choice Tasks
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“relative to control rats, rats with lesions to DMS lever-press at a lower rate and are less able to decrease their press rate when reinforcers are omitted or devalued (Yin et al., 2005)”
§ Habits without Values > Results > Effects of Striatal Lesions on Habit Formation and Behavioral Flexibility
What the sources say
- Not checked yetcites Yin et al., 2005
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“rats with lesions to DLS lever-press at a similar rate to controls and are more successful than controls at adapting their press rate to omitted or devalued reinforcement (Yin, Knowlton, & Balleine, 2004, 2006).”
§ Habits without Values > Results > Effects of Striatal Lesions on Habit Formation and Behavioral Flexibility
What the sources say
- Not checked yetcites Yin, Knowlton, & Balleine, 2004, 2006
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This computational account in which habits are understood as value-free stimulus-response associations therefore provides a closer match to classic psychological theories of habits, an account for classic behavioral data on habit formation, and a novel framework for understanding additional behavioral phenomena.”
§ Habits without Values > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearIt may state two separate claims as one.
In the paper
“In one recent example, rats performing a dynamic two-armed bandit task exhibited behavioral patterns consistent with both reinforcement-seeking (i.e., being more likely to select a recently reinforced action), as well as with choice perseveration (i.e., being more likely to select a recently chosen action).”
§ Habits without Values > Results > Perseverative Behavior in Sequential Choice Tasks
What the sources say
- Not checked yetcites Miller et al., in prep
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“The extra computational complexity associated with the goal-directed system was a source of uncertainty, leading the arbiter to favor the habitual system in well-learned environments.”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites Daw et al., 2005
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“This arbiter is similar to mechanisms found in computational accounts mapping habitual/goal-directed control onto model-free/model-based reinforcement learning.”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites Daw et al., 2005
- Not checked yetcites S. W. Lee et al., 2014
- Not checked yetcites Keramati et al., 2011
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“A large and influential body of computational work is built on the assumption that habitual control arises from model-free reinforcement learning algorithms (Dolan & Dayan, 2013; O’Doherty, Lee, & McNamee, 2015).”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites Dolan & Dayan, 2013
- Not checked yetcites O’Doherty, Lee, & McNamee, 2015
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Subsequent accounts have developed arbiters that consider proxies for this uncertainty (S. W. Lee et al., 2014), or other possible advantages of habitual over goal-directed control, such as response time (Keramati et al., 2011).”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites S. W. Lee et al., 2014
- Not checked yetcites Keramati et al., 2011
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In rodents, goal-directed behavior results in greater activity in OFC and DMS, while habitual behavior results in greater activity in DLS (Gremel & Costa, 2013).”
§ Habits without Values > Discussion > Tensions in Neuroscientific Data
What the sources say
- Not checked yetcites Gremel & Costa, 2013
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the value-free mechanisms we have proposed for habits by no means preclude a role for value or motivational state (e.g., hunger or satiety) in habit learning.”
§ Habits without Values > Discussion > Implications
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: In the model, control of behavior is allocated on each trial to either the habitual or the goal-directed system by an arbiter.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These may modulate the strengthening of habit associations either directly (e.g., through a feedback mechanism that influences the S-R association) or indirectly (e.g., by influencing the vigor of an action, which in turn results in greater associative strengths).”
§ Habits without Values > Discussion > Implications
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- The authors state this as an assumption: In the model, control of behavior is allocated on each trial to either the habitual or the goal-directed system by an arbiter.
None of the sources cited for this claim have been checked yet.
In the paper
“In human subjects, goal-directed value signals during outcome devaluation have been identified in vmPFC (Valentin, Dickinson, & O’Doherty, 2007), and similar signals during contingency degradation have been identified both in vmPFC and in the caudate nucleus, a homolog of DMS (Tanaka et al., 2008).”
§ Habits without Values > Discussion > Tensions in Neuroscientific Data
What the sources say
- Not checked yetcites Valentin, Dickinson, & O’Doherty, 2007
- Not checked yetcites Tanaka et al., 2008
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our proposed framework re-maps habits to the perseveration kernel and provides an account of how that kernel might plausibly operate in tandem with a goal-directed controller so as to account for behaviors that have previously been described by RL models.”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These data indicate that dopamine neurons are unlikely to play a role in a uniquely model-free control system, but instead have access to model-based information and play a role in model-based control.”
§ Habits without Values > Discussion > Tensions in Neuroscientific Data
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In effect, we are showing that the model-free component of some of these previous formalizations might not be necessary and that an elaboration of this perseveration kernel actually serves as a better model of habits.”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In sum, considerable evidence supports the idea that anatomically separate goal-directed and habitual controllers exist in the brain, and that either controller can be responsible for a given action.”
§ Habits without Values > Discussion > Tensions in Neuroscientific Data
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“habits are promoted in situations where the contingencies between actions and outcomes are weak (Dickinson, 1985).”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites Dickinson, 1985
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We first propose severing the tie between habits and model-free RL, and instead defining a category of behaviors that are “value-free” and therefore distinct from either type of RL computation.”
§ Habits without Values > Discussion > A Proposed Realignment
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
In the paper
“In the initial formalization of this idea (Daw et al., 2005), each system determined the uncertainty in its value estimates, and the arbiter selected the system with less overall uncertainty.”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites Daw et al., 2005
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“In addition to utilizing different mechanisms for value-based control and for arbitrating between the value-based and value-free controllers, our model also differs from these corticostriatal models in the level of analysis at which it is described. While this limits the level of detail with which our model can engage neurobiological data, it greatly facilitates engagement with a wide variety of”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites Frank, 2015
- Not checked yetcites Frank & Badre, 2015
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Beyond providing a potentially more parsimonious account of previous behavioral results”
§ Habits without Values > Conclusions
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: In the model, control of behavior is allocated on each trial to either the habitual or the goal-directed system by an arbiter.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“such a realignment may offer a better account of extant neural findings, including the fact that structures associated with model-free and model-based computations (i.e., value-based computations) tend to overlap, whereas lesion/inactivation studies have revealed clear dissociations between structures associated with goal-directed behavior versus (potentially value-free) habits.”
§ Habits without Values > Conclusions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearIt may state two separate claims as one.
- The authors state this as an assumption: In the model, control of behavior is allocated on each trial to either the habitual or the goal-directed system by an arbiter.
The paper relies on this without support.
The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
“To illustrate the behavior of the model and the dynamics of its various internal variables, we simulated behavior in a probabilistic reversal learning task (Fig 3).”
The paper relies on this without support.
The model's goal-directed controller corresponds to the dorsomedial striatum (DMS) and its habitual controller corresponds to the dorsolateral striatum (DLS).
“We lesioned the goal-directed controller (DMS) or habitual controller (DLS) in our model”
“Lesions to our model’s goal-directed and habitual controllers thus reproduce behavioral patterns typical of DMS and DLS lesioned rats”
The authors state this as an assumption.
In the model, control of behavior is allocated on each trial to either the habitual or the goal-directed system by an arbiter.
“In our model, control of behavior is allocated on each trial to either the habitual or the goal-directed system by an arbiter.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Supporting this, computational studies have proposed two parallel forms of reinforcement learning, model-based and model-free (or value-free) learning corresponding to goal-directed and habitual strategies, respectively, that develop concurrently during behavior acquisition2,41,42.”
From Dissociable roles of prefrontal plasticity in decision-making strategy and execution of habitual behavior (2026) · cites this paper as [42]
Experimental — accuracy not yet measuredIn the citing paper
“With overtraining, the habitual system simplifies decision-making by shifting to a stimulus-response mode of behavior51, making decisions potentially value-less and inflexible but less cognitively demanding52.”
From Revealing abrupt transitions from goal-directed to habitual behavior (2026) · cites this paper as [52]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Training reinforcement probability differs between Methods and Results
MajorThe Methods specify a 25% pellet probability during omission training, and say devaluation training was identical. The Results describe a 50% pellet probability for both training phases.
“In the first phase, performance of one action (Press Lever) resulted in a reinforcer of one type (Pellet) 25% of the time”
“The agent was initially trained in an environment where performance of Action A (“press lever”) was followed by a reinforcer of one type (“food pellet”) 50% of the time”
Habit-weight update uses a different basis function
MinorThe habit-strength function uses basis functions γi, so its derivative with respect to weight ci is γi. The stated weight-update equation instead substitutes χi, which is not the defined habit basis function.
“=ct(i)+αH⋅∑a′((δa′,at−H(a′))⋅χi(a′))”
“11Ht(a)=∑i=130ct(i)⋅γi(a)γi(a)=e−(a−5i)22⋅52”
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
James, W. (1891) · Personality Culture Society · doi:10.1037/11059-000 · Cited by 25,876
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256
The principles of psychology, Vol I.
James, W. (1890) · doi:10.1037/10538-000 · Cited by 4,514
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
The role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402
How are habits formed: modelling habit formation in the real world
Lally, P., van Jaarsveld, C. H. M., Potts, H. W. W., & Wardle, J. (2010) · European Journal of Social Psychology · doi:10.1002/ejsp.674 · Cited by 2,036
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
Flagged first, then by citations
1 of the 136 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Items 101–200 of 277
Residual Information of Previous Decision Affects Evidence Accumulation in Current Decision
Olianezhad, F., Zabbah, S., Tohidi-Moghaddam, M., & Ebrahimpour, R. (2019) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2019.00009 · Cited by 14
Humans are primarily model-based learners in the two-stage taskPreprint
da Silva, C. F., & Hare, T. A. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/682922 · Cited by 17
Dual Decision Processes: Retrieving Preferences When Some Choices Are Automatic
Cerigioni, F. (2021) · Journal of Political Economy · doi:10.1086/713730 · Cited by 18
Spontaneous instrumental avoidance learning in social contexts
Mennella, R., Bavard, S., Mentec, I., & Grèzes, J. (2022) · Scientific Reports · doi:10.1038/s41598-022-22334-6 · Cited by 14
Working memory capacity estimates moderate value learning for outcome-irrelevant features
Ben-Artzi, I., Luria, R., & Shahar, N. (2022) · Scientific Reports · doi:10.1038/s41598-022-21832-x · Cited by 11
Taylor, V. A., Moseley, I. H., Sun, S., Smith, R. S., Roy, A., Ludwig, V. U., & Brewer, J. A. (2021) · Journal of Behavioral Addictions · doi:10.1556/2006.2021.00020 · Cited by 14
Time pressure changes how people explore and respond to uncertaintyPreprint
Wu, C. M., Schulz, E., Pleskac, T. J., & Speekenbrink, M. (2021) · doi:10.31234/osf.io/dsw7q · Cited by 14
Dissociable habits of response preparation versus response initiation
Du, Y., & Haith, A. M. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02215-4 · Cited by 13
Acetylcholine demixes heterogeneous dopamine signals for learning and moving
Jang, H. J., Ward, R. M., Golden, C. E. M., & Constantinople, C. M. (2026) · Nature Neuroscience · doi:10.1038/s41593-026-02227-x · Cited by 20
Model based planners reflect on their model-free propensities
Moran, R., Keramati, M., & Dolan, R. J. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008552 · Cited by 14
DREAM Architecture: a Developmental Approach to Open-Ended Learning in RoboticsPreprint
Doncieux, S., Bredèche, N., Goff, L. K. L., Girard, B., Coninx, A., Sigaud, O., Khamassi, M., Díaz-Rodríguez, N., Filliat, D., Hospedales, T. M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.06223 · Cited by 18
Re-Thinking the Organization of Cortico-Basal Ganglia-Thalamo-Cortical Loops
Baladron, J., & Hamker, F. H. (2023) · Cognitive Computation · doi:10.1007/s12559-023-10140-9 · Cited by 12
The two‐step task, avoidance, and OCD
Geramita, M. A., Yttri, E. A., & Ahmari, S. E. (2020) · Journal of Neuroscience Research · doi:10.1002/jnr.24594 · Cited by 12
Nestor, L. J., & Ersche, K. D. (2022) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2022.08.014 · Cited by 11
Tuzsus, D., Brands, A. M. A., Pappas, I., & Peters, J. (2024) · Computational Brain & Behavior · doi:10.1007/s42113-024-00202-y · Cited by 10
Understanding dual process cognition via the minimum description length principle
Moskovitz, T., Miller, K. J., Sahani, M., & Botvinick, M. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012383 · Cited by 21
Self-citationO’Reilly, R. C., Nair, A., Russin, J., & Herd, S. A. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00380 · Cited by 14
Rationalization as representational exchange: Scope and mechanism
Cushman, F. A. (2020) · Behavioral and Brain Sciences · doi:10.1017/s0140525x19003261 · Cited by 11
Investigating the origin and consequences of endogenous default options in repeated economic choices
Couto, J., van Maanen, L., & Lebreton, M. P. (2020) · PLoS ONE · doi:10.1371/journal.pone.0232385 · Cited by 11
Representational exchange in human social learning: Balancing efficiency and flexibilityPreprint
Wu, C. M., Vélez, N., & Cushman, F. (2021) · doi:10.31234/osf.io/rm52c · Cited by 21
Hoxha, I., Sperber, L., & Palminteri, S. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2422144122 · Cited by 12
Multiscale Computation and Dynamic Attention in Biological and Artificial Intelligence
Badman, R. P., Hills, T. T., & Akaishi, R. (2020) · Brain Sciences · doi:10.3390/brainsci10060396 · Cited by 9
Rutherford, L. G., & Milton, A. L. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.104514 · Cited by 10
Impaired selection of a previously ignored singleton: Evidence for salience map plastic changes
Turatto, M., & Valsecchi, M. (2021) · Quarterly Journal of Experimental Psychology · doi:10.1177/17470218211047859 · Cited by 9
Interaction between habits as action sequences and goal-directed behavior under time pressure
Frölich, S., Esmeyer, M., Endraß, T., Smolka, M. N., & Kiebel, S. J. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.996957 · Cited by 10
Modelling cognitive flexibility with deep neural networks
Sandbrink, K. J., & Summerfield, C. (2024) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2024.101361 · Cited by 14
Habit and Automaticity in Medical Alert Override: Cohort Study
Wang, L., Goh, K. H., Yeow, A. Y. K., Poh, H. M. N., Li, K., Yeow, J. J. L., Tan, G. Y. H., & Soh, C. (2022) · Journal of Medical Internet Research · doi:10.2196/23355 · Cited by 9
The post-pandemic era: will physical distancing be a perceived way of life?
Khan, S. Q., AlHumaid, J., Farooqi, F. A., Gad, M. M., Alhareky, M. S., Alonaizan, F. A., & Al‐Harbi, F. A. (2021) · F1000Research · doi:10.12688/f1000research.52779.1 · Cited by 8
Zhang, C., Spelt, H., van Wissen, A., Lakens, D., & IJsselsteijn, W. A. (2022) · Health Psychology · doi:10.1037/hea0001199 · Cited by 8
Handel, S. N., & Smith, R. J. (2023) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.889 · Cited by 10
The Specification of a Computational Model of Physical Activity Habit
Rebar, A. L., Lagoa, C. M., Gardner, B., & Conroy, D. E. (2024) · Exercise and Sport Sciences Reviews · doi:10.1249/jes.0000000000000340 · Cited by 8
Velázquez-Vargas, C. A., Daw, N. D., & Taylor, J. A. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012471 · Cited by 10
Computational Model of the Transition from Novice to Expert Interaction Techniques
Bailly, G., Khamassi, M., & Girard, B. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3505557 · Cited by 11
Model based control can give rise to devaluation insensitive choice
Garrett, N., Allan, S., & Daw, N. D. (2023) · Addiction Neuroscience · doi:10.1016/j.addicn.2023.100070 · Cited by 7
Signatures of heuristic-based directed exploration in two-step sequential decision task behaviour
Brands, A. M. A., Mathar, D., & Peters, J. (2025) · Computational Psychiatry · doi:10.5334/cpsy.101 · Cited by 6
A habit and working memory model as an alternative account of human reward-based learning
Collins, A. G. E. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02340-0 · Cited by 12
Political reinforcement learners
Schulz, L., & Bhui, R. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.12.001 · Cited by 8
Reciprocal relationships between depressive symptoms and academic amotivation in college students
Cui, G., Shen, B., & Bo, J. (2024) · Educational Psychology · doi:10.1080/01443410.2024.2354688 · Cited by 7
Disentangling choice value and choice conflict in sequential decisions under risk
Fontanesi, L., Shenhav, A., & Gluth, S. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010478 · Cited by 7
Self-citationColas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6
The successor representation subserves hierarchical abstraction for goal-directed behavior
Wientjes, S., & Holroyd, C. B. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011312 · Cited by 8
A behavioral model of adaptation
Apesteguía, J., Ballester, M. A., & Çuhadaroğlu, T. (2023) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2023.01.008 · Cited by 9
On the Interest of Extending Internet of Behaviors to Internet of Habits: A Preliminary Exploration
Abri, K. A. S. A., Mastouri, M., Bouyahia, Z., Haddad, H., Yasar, A.-U.-H., & Shakshuki, E. (2023) · Procedia Computer Science · doi:10.1016/j.procs.2023.09.019 · Cited by 6
Emotional Mechanisms of Behavior Change: Existing Techniques, Best Practices, and a New Approach
Shiota, M. N., Vornlocher, C., & Jia, L. (2023) · Policy Insights from the Behavioral and Brain Sciences · doi:10.1177/23727322231195907 · Cited by 8
A Sequential Sampling Approach to the Integration of Habits and Goals
Zhang, C., van Wissen, A., Dotsch, R., Lakens, D., & IJsselsteijn, W. A. (2024) · Computational Brain & Behavior · doi:10.1007/s42113-024-00199-4 · Cited by 7
Cognitive computational model reveals repetition bias in a sequential decision-making task
Legler, E., Rivera, D. C., Schwöbel, S., Wagner, B., & Kiebel, S. J. (2025) · Communications Psychology · doi:10.1038/s44271-025-00271-0 · Cited by 7
An interdisciplinary synthesis of research on understanding and promoting well‐doing
Lieder, F., Prentice, M. S., & Corwin‐Renner, E. R. (2022) · Social and Personality Psychology Compass · doi:10.1111/spc3.12704 · Cited by 8
Studying human habit formation through motor sequence learning
Grundmann, C. C., Arndt, V. A., Ebrahimi, C., Musial, M. P. M., Bode, E. L., Schlagenhauf, F., & Endraß, T. (2025) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-025-01300-5 · Cited by 6
Distinct cortico-striatal compartments drive competition between adaptive and automatized behavior
Barnett, W. H., Kuznetsov, A. S., & Lapish, C. C. (2023) · PLoS ONE · doi:10.1371/journal.pone.0279841 · Cited by 6
Defining and Measuring Habits Across Different Fields of Research
Watson, P. (2024) · Springer eBooks · doi:10.1007/978-3-031-55889-4_1 · Cited by 5
Tanner, C., Schmocker, D., Katsarov, J. W., & Christen, M. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3838222 · Cited by 5
Explaining the effectiveness of fear extinction through latent-cause inferencePreprint
Song, M., Jones, C. E., Monfils, M., & Niv, Y. (2021) · doi:10.31234/osf.io/2fhr7 · Cited by 5
Belief updates, learning and adaptive decision makingPreprint
Frömer, R., & Nassar, M. R. (2023) · doi:10.31234/osf.io/qndba · Cited by 5
Delayed rewards weaken human goal directed actions
Perez, O. D., & Urcelay, G. P. (2025) · npj Science of Learning · doi:10.1038/s41539-025-00325-2 · Cited by 4
Getting stuck in a rut as an emergent feature of a dynamic decision-making system
Warburton, M., Brookes, J., Hasan, M., Leonetti, M., Doğar, M. R., Wang, H., Cohn, A. G., Mushtaq, F., & Mon-Williams, M. A. (2024) · Royal Society Open Science · doi:10.1098/rsos.231550 · Cited by 4
Distributional dual-process model predicts strategic shifts in decision-making under uncertainty
Hu, M., Don, H. J., & Worthy, D. A. (2025) · Communications Psychology · doi:10.1038/s44271-025-00249-y · Cited by 4
Action repetition biases choice in context-dependent decision-making
Wagner, B., Wolf, H. B., & Kiebel, S. J. (2025) · Communications Psychology · doi:10.1038/s44271-025-00363-x · Cited by 8
Using recurrent neural network to estimate irreducible stochasticity in human choice behavior
Ger, Y., Shahar, M., & Shahar, N. (2024) · eLife · doi:10.7554/elife.90082 · Cited by 15
Taylor, V. A., Smith, R. S., & Brewer, J. A. (2022) · Sensors · doi:10.3390/s22145131 · Cited by 4
Multi-Plasticities: Distinguishing Context-Specific Habits from Complex Perseverations
Banavar, N. V., & Bornstein, A. M. (2024) · Springer eBooks · doi:10.1007/978-3-031-55889-4_5 · Cited by 4
Shaping Model-Free Habits with Model-Based Goals.
Krueger, P. M., & Griffiths, T. L. (2018) · eScholarship (California Digital Library) · Cited by 4
Common roles for serotonin in rats and humans for computations underlying flexible decision-makingPreprint
Luo, Q., Kanen, J. W., Bari, A., Skandali, N., Langley, C., Knudsen, G. M., Alsiö, J., Phillips, B. U., Sahakian, B. J., Cardinal, R. N., et al. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.02.15.527569 · Cited by 4
Revealing abrupt transitions from goal-directed to habitual behaviorPreprint
Moore, S., Wang, Z., Zhu, Z., Sun, R., Lee, Á., Charles, A. S., & Kuchibhotla, K. V. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.07.05.547783 · Cited by 4
Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learningPreprint
Venditto, S. J. C., Miller, K. J., Brody, C. D., & Daw, N. D. (2024) · eLife · doi:10.7554/elife.97612.2 · Cited by 4
Self-citationRevealing abrupt transitions from goal-directed to habitual behavior
Moore, S., Wang, Z., Zhu, Z., Wang, J., Sun, R., Lee, Y. A., Charles, A., & Kuchibhotla, K. V. (2026) · Nature Communications · doi:10.1038/s41467-026-71048-0 · Cited by 3
Learning to Move and Plan like the Knight: Sequential Decision Making with a Novel Motor Mapping
Velázquez-Vargas, C. A., & Taylor, J. A. (2025) · Computational Brain & Behavior · doi:10.1007/s42113-025-00245-9 · Cited by 4
Sinchai, A., Koiwanit, J., Chatmarathong, A., & Filimonau, V. (2026) · Journal of Hospitality and Tourism Management · doi:10.1016/j.jhtm.2026.101491 · Cited by 3
Behavior change after context disruption: Opportunities and pitfalls
McMillan, G., Milyavskaya, M., & Burns, R. J. (2023) · Social and Personality Psychology Compass · doi:10.1111/spc3.12845 · Cited by 3
Dorsal prefrontal cortex drives perseverative behavior in mice
Lebedeva, A., Wang, Y., Funnell, L., Terry, B., Oh, J. K., Miller, K. J., & Harris, K. D. (2026) · Nature Communications · doi:10.1038/s41467-026-71664-w · Cited by 5
Self-citationA Bio-Inspired Dopamine Model for Robots with Autonomous Decision-Making
Gómez, M. M., Burguete-Alventosa, J., Álvarez-Arias, S., Malfáz, M., & Salichs, M. A. (2024) · Biomimetics · doi:10.3390/biomimetics9080504 · Cited by 3
Contribution of amygdala to dynamic model arbitration under uncertainty
Woo, J. H., Costa, V. D., Taswell, C. A., Rothenhoefer, K. M., Averbeck, B. B., & Soltani, A. (2025) · Nature Communications · doi:10.1038/s41467-025-66745-1 · Cited by 3
Sosa, R. (2022) · Adaptive Behavior · doi:10.1177/10597123221097451 · Cited by 5
Zafar, M. A., Gill, A. R., Nawaz, M. A., & Ahmad, T. I. (2024) · THE ASIAN BULLETIN OF GREEN MANAGEMENT AND CIRCULAR ECONOMY · doi:10.62019/abgmce.v4i1.88 · Cited by 3
Lin, R., Yan, Y., Chen, Z., Huang, C., Zhao, J., Chen, M., & Li, H. (2025) · BMC Geriatrics · doi:10.1186/s12877-025-05684-4 · Cited by 3
Habit-Based and Goal-Directed Systems: Knowledge Transfer in Individual and Social Learning
Lowe, R. J. (2020) · Studies in applied philosophy, epistemology and rational ethics · doi:10.1007/978-3-030-37305-4_10 · Cited by 3
Reward prediction error does not explain movement selectivity in DMS-projecting dopamine neuronsPreprint
Lee, R. S., Mattar, M. G., Parker, N. F., Witten, I. B., & Daw, N. D. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/447532 · Cited by 3
Humans are primarily model-based and not model-free learners in the two-stage taskPreprint
Hare, T. A., & da Silva, C. F. (2019) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-184469 · Cited by 6
Bavard, S., Rustichini, A., & Palminteri, S. (2020) · doi:10.1101/2020.07.28.224642 · Cited by 3
Testing models of context-dependent outcome encoding in reinforcement learningPreprint
Hayes, W. M., & Wedell, D. H. (2022) · doi:10.31234/osf.io/3mn5u · Cited by 3
Hamroun, S., Lebreton, M. P., & Palminteri, S. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/2uw85 · Cited by 3
Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learningPreprint
Venditto, S. J. C., Miller, K. J., Brody, C. D., & Daw, N. D. (2024) · eLife · doi:10.7554/elife.97612.1 · Cited by 3
Self-citationA computational principle of habit formationPreprint
Lakshminarasimhan, K. J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.10.12.618033 · Cited by 3
Multiple and subject-specific roles of uncertainty in reward-guided decision-makingPreprint
Paunov, A. M., L’Hôtellier, M., Guo, D., He, Z., Yu, A. J., & Meyniel, F. (2024) · eLife · doi:10.7554/elife.103363 · Cited by 5
A corticostriatal learning mechanism linking excess striatal dopamine and auditory hallucinationsPreprint
Lakshminarasimhan, K. J., Buck, J., Kellendonk, C., & Horga, G. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.03.18.643990 · Cited by 3
Baumann, C., Schlegelmilch, R., & von Helversen, B. (2024) · Cognition · doi:10.1016/j.cognition.2024.106001 · Cited by 2
Model-based algorithms shape automatic evaluative processing
Melnikoff, D. E., & Kurdi, B. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2417068122 · Cited by 2
Hybrid neural–cognitive models reveal how memory shapes human reward learning
Eckstein, M. K., Summerfield, C., Daw, N. D., & Miller, K. J. (2026) · Nature Human Behaviour · doi:10.1038/s41562-025-02324-0 · Cited by 19
Self-citationChai, J., Ruan, X., & Huang, J. (2021) · Cognitive Computation · doi:10.1007/s12559-021-09950-6 · Cited by 2
WILL PHYSICAL DISTANCING BE THE NEW PERCEIVED WAY OF LIFE IN THE POST-COVID-19 PANDEMIC ERA?
Khan, S. Q., Moheet, I. A., Farooqi, F. A., Khan, N. B., Wahab, S., Haider, I., Al-Shayea, N., & Ali, S. (2023) · Journal of Medical Sciences · doi:10.52764/jms.23.31.1.11 · Cited by 2
Resource-rational psychopathology.
Bari, B. A., & Gershman, S. J. (2024) · Behavioral Neuroscience · doi:10.1037/bne0000600 · Cited by 4
Reference-point dependent reinforcement learning in humans and rats
Ferguson, L. A., Soukupova, M., Bouret, S., Palminteri, S., & Parkes, S. L. (2026) · Nature Communications · doi:10.1038/s41467-026-73623-x · Cited by 3
Neural and behavioral signatures of policy compression in cognitive control
Liu, S., Kikumoto, A., Badre, D., & Gershman, S. J. (2025) · Cerebral Cortex · doi:10.1093/cercor/bhaf223 · Cited by 2
Hamker, F. H., Baladron, J., & Janssen, L. K. (2025) · Trends in Neurosciences · doi:10.1016/j.tins.2025.09.006 · Cited by 4
Integrating neuro-psychological habit research into consumer choice models
Webb, R., Fong, J., Landry, P., Levine, J., Wellsjo, A., Natan, O., Mazar, A., Zhao, C., Lally, P. J., de Wit, S., et al. (2025) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2025.10.002 · Cited by 2
Sookud, S., Martin, I. M., Gillan, C. M., & Wise, T. (2025) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2025.10.005 · Cited by 2
Ebrahimi, C., Musial, M. P. M., Doñamayor, N., Prychynenko, D. S., Bode, E. L., Spanagel, R., Heinz, A., Deserno, L., Smolka, M. N., Ebner‐Priemer, U., et al. (2025) · Translational Psychiatry · doi:10.1038/s41398-025-03660-5 · Cited by 3
Fentanyl reinforcement history has sex-specific effects on multi-step decision-making
Garr, E., Cheng, Y., Dong, A., & Janak, P. H. (2026) · Neuropharmacology · doi:10.1016/j.neuropharm.2026.110860 · Cited by 2
Choice perseverance underlies pursuing a hard-to-get target in an avatar choice task
Sugawara, M., & Katahira, K. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.924578 · Cited by 2
Theory-based Habit Modeling for Enhancing Behavior PredictionPreprint
Zhang, C., Vanschoren, J., van Wissen, A., Lakens, D., de Ruyter, B., & IJsselsteijn, W. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.01637 · Cited by 2
Kanen, J. W., Luo, Q., Yellowlees, R., Arntz, F. E., Christmas, D. M., Sahakian, B. J., Apergis-Schoute, A. M., Cardinal, R. N., & Robbins, T. W. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.04.05.437619 · Cited by 2
Metadata
- Authors
- Miller, K. J., Shenhav, A., & Ludvig, E. A.
- Affiliations
- Princeton University (US)
- Venue
- Psychological Review
- Published
- 2019-03
- Volume
- 126
- Issue
- 2
- Pages
- 292-311
- ISSN
- 0033-295X (ISSN-L)
- 1939-1471
- PMC
- PMC6548181
- OpenAlex
- W2949941589
- Cited by
- 280
- References
- 136
- Citations by year
- 2026: 32 · 2025: 38 · 2024: 52 · 2023: 29 · 2022: 41 · 2021: 41 · 2020: 24 · 2019: 14
- FWCI
- 17.32
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Psychological Review (publishedVersion, other-oa)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Warwick Research Archive Portal (University of Warwick) (submittedVersion)
- Funders
- National Institutes of Health (T-32 MH065214)
- Princeton University
- Sponsor name not included
- National Institute of General Medical Sciences (P20GM103645)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Psychology › Applied Psychology › Behavioral Health and Interventions
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- habit, stimulus-response learning, model-free reinforcement learning, outcome devaluation, contingency degradation, reinforcement schedules, perseveration, instrumental behavior, habit formation
- MeSH terms
- Animals, Behavior, Animal, Decision Making, Goals, Habits, Humans, Models, Psychological, Reinforcement, Psychology, Prefrontal Cortex
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
