The DOI matches its Crossref record; 135 of the 136 references it cites were found in a registry or an index; none of the 99 sources its claims cite is checked for support yet.
Habits without values.
Open accessPublished version of a preprint
Abstract
Habits form a crucial component of behavior. In recent years, key computational models have conceptualized habits as arising from model-free reinforcement learning mechanisms, which typically select between available actions based on the future value expected to result from each. Traditionally, however, habits have been understood as behaviors that can be triggered directly by a stimulus, without requiring the animal to evaluate expected outcomes. Here, we develop a computational model instantiating this traditional view, in which habits develop through the direct strengthening of recently taken actions rather than through the encoding of outcomes. We demonstrate that this model accounts for key behavioral manifestations of habits, including insensitivity to outcome devaluation and contingency degradation, as well as the effects of reinforcement schedule on the rate of habit formation. The model also explains the prevalent observation of perseveration in repeated-choice tasks as an additional behavioral manifestation of the habit system. We suggest that mapping habitual behaviors onto value-free mechanisms provides a parsimonious account of existing behavioral and neural data. This mapping may provide a new foundation for building robust and comprehensive models of the interaction of habits with other, more goal-directed types of behaviors and help to better guide research into the neural mechanisms underlying control of instrumental behavior more generally. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper develops a computational model in which habits are value-free stimulus-response associations strengthened by repetition, rather than model-free reinforcement learning. It shows this model reproduces key habit phenomena, including insensitivity to outcome devaluation and contingency degradation, reinforcement-schedule effects on habit formation, lesion-like dissociations, and perseveration.
Key findings
- A value-free habitual controller, combined with a model-based goal-directed controller and an arbiter, reproduces overtraining-induced inflexibility: after longer training, simulated agents continued lever-pressing under omission and devaluation, whereas shorter training allowed reversal. Training duration was varied from 100 to 2000 trials in intervals of 100 for omission and from 10 to 2000 trials in intervals of 100 for devaluation.
- Simulated agents trained on variable-interval (VI6: pellets baited every 6 s, on average 10/min) lever-pressed at a moderate rate and habitized early, whereas agents trained on variable-ratio (VR10: 10% chance per press) lever-pressed at a high rate and habitized much later.
- Simulated lesions reproduced rodent striatal patterns: decreasing goal-directed parameters (DMS-like) produced a low press rate unaffected by omission or devaluation; decreasing habitual parameters (DLS-like) produced a high press rate that adapted to both manipulations. Control agents adapted after limited training (5,000 trials) but not after extensive training (15,000 trials).
- In a dynamic two-armed bandit task (10,000 trials; 50 model copies; reinforcer probabilities changed by independent Gaussian random walks, SD = 0.15), the model reproduced both reinforcement-seeking and value-free perseveration; replacing the habitual component with model-free RL failed to reproduce the perseverative pattern.
Methods
- Computational model with three modules: a habitual controller tracking recency-weighted action history \(H\), a goal-directed controller using model-based RL to learn separate reinforcer expectations \(R\) and utilities \(U\), and an arbiter weighting control by action-outcome contingency \(g\) and overall habitization \(h\).
- Simulated task environments included reversal learning, omission contingency, outcome devaluation, free-operant VR/VI schedules, simulated lesions, and a two-armed bandit. Most simulations used a single state with two actions; free-operant simulations used continuous press rates from 0 to 150 presses/min with function approximation and an effort cost quadratic in press rate.
- Analyses averaged simulated performance across agents and, for the two-armed bandit, used logistic regression quantifying influences of past actions, reinforcers, and their interaction on future choice.
Limitations
- The paper states that the model is described at an abstract computational level, limiting engagement with neurobiological detail, and that a neurobiologically detailed implementation is a critical next step.
- The paper states that the arbiter’s conditions require future research, that the model is agnostic about whether bypassing model-free RL entirely is appropriate, and that the form of value/motivational modulation of habit learning remains unresolved.
- Evident from the text: the findings come from simulations rather than new empirical experiments, and many simulations use simplified single-state, two-action environments with specific parameter and utility choices, so generality across richer environments is not established.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 99 cited sources checked · 0 supported · 0 flagged
None of the sources cited for this claim have been checked yet.
In the paper
“as well as the effects of reinforcement schedule on the rate of habit formation.”
§ Habits without Values > Abstract
What the sources say
- Not checked yetcites Adams, 1982
- Not checked yetcites Dickinson, 1985
- Not checked yetcites Gremel & Costa, 2013
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We demonstrate that this model accounts for key behavioral manifestations of habits, including insensitivity to outcome devaluation and contingency degradation”
§ Habits without Values > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The model also explains the prevalent observation of perseveration in repeated-choice tasks as an additional behavioral manifestation of the habit system.”
§ Habits without Values > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
None of the sources cited for this claim have been checked yet.
In the paper
“This value-free habit mechanism accounts for key findings in the animal learning literature that dissociate habitual and goal-directed actions, namely the tendency for an animal to continue performing a previously learned action when that action is no longer predictive of the reinforcing outcome (contingency degradation; Hammond, 1980) or when the predicted outcome ceases to be desired by the subj”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Hammond, 1980
- Not checked yetcites Adams & Dickinson, 1981
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As a result, when the reversal occurs, the agent continues to select Action A for an extended period, past the point where the goal-directed controller has learned that Action B is more likely to be reinforced (compare Figures 3B and 3E).”
§ Habits without Values > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“In addition, this model provides what is, to our knowledge, the first computational account of the difference in rate of habit formation under variable-interval and variable-ratio schedules of reinforcement (Adams, 1982; Dickinson, 1985; Gremel & Costa, 2013).”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Adams, 1982
- Not checked yetcites Dickinson, 1985
- Not checked yetcites Gremel & Costa, 2013
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Initially, behavior is driven by the goal-directed controller, which gradually learns the relative reinforcement rates (Figure 3B) and thus increasingly selects Action A.”
§ Habits without Values > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In the following sections we demonstrate that this model can capture all of the key diagnostic features of habitual behavior previously identified in animal behavior, including sensitivity to repetition frequency, reinforcement schedule, and selective modulation by lesions to one of two dissociable neural circuits.”
§ Habits without Values > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition to demonstrating the different kinds of learning that drive each of these controllers, this example demonstrates that our model captures the observation that behavioral control in a novel environment tends to evolve from goal-directed to habitual (i.e., habits form from actions that were originally selected in a value-based manner).”
§ Habits without Values > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A “control” agent, with intact habitual and goal-directed controllers, adopted a high press rate that adapted to both manipulations when given limited training, but did not adapt to either following extensive training.”
§ Habits without Values > Results > Effects of Striatal Lesions on Habit Formation and Behavioral Flexibility
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 21
- Probably not a substantial claim. 18
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“whereas agents trained on VR schedules lever-pressed at a high rate and habitized much later in training (Figure 4, right).”
§ Habits without Values > Results > Effects of Reinforcement Schedule on Habit Formation
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we found that simulated agents trained on VI schedules lever-pressed at a moderate rate and habitized early in training (Figure 4, left)”
§ Habits without Values > Results > Effects of Reinforcement Schedule on Habit Formation
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Model-free algorithms typically operate by learning the expected future reward associated with each possible action or state, relying crucially on these value representations.”
§ Habits without Values > Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Goal-directed behaviors are understood to be driven by consideration of the outcomes that they are likely to bring about (i.e. “action-outcome” representations).”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Balleine & O’Doherty, 2010
- Not checked yetcites Dickinson, 1985
- Not checked yetcites Dolan & Dayan, 2013
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“This category of value-free habits has been maintained in modern theorizing on habits, where it has been referred to as “direct cuing” of behavior, as distinct from value-based forms of habits (“motivated cueing”; Wood & Neal, 2007; Wood & Rünger, 2016).”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Wood & Neal, 2007
- Not checked yetcites Wood & Rünger, 2016
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Modern reinforcement learning (RL) has elaborated this law into a set of computational algorithms, according to which actions are selected based on cached values learned from previous experience (Daw, Niv, & Dayan, 2005; Dolan & Dayan, 2013; Sutton & Barto, 1998).”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Daw, Niv, & Dayan, 2005
- Not checked yetcites Dolan & Dayan, 2013
- Not checked yetcites Sutton & Barto, 1998
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This class of computations focuses only on potential rewards, ignoring all reward-unrelated elements of one’s environment (discussed below); it is therefore referred to as “model-free” RL.”
§ Habits without Values > Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“This formulation for habits has become so prevalent that the terms “habit” and “model-free” are now used interchangeably in much of the computational literature (Dolan & Dayan, 2013; Doll, Simon, & Daw, 2012).”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Dolan & Dayan, 2013
- Not checked yetcites Doll, Simon, & Daw, 2012
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
In the paper
“This law holds that an action that has been taken often in the past is likely to be repeated in the future, independent of its past consequences; in other words, it describes habits as a form of perseveration.”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Thorndike, 1911
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“A large and growing body of work indicates that these behaviors depend on different sets of computations and distinct underlying neural circuits, suggesting that separable goal-directed and habitual systems implement fundamentally different strategies for the control of behavior”
§ Habits without Values > Introduction
What the sources say
- Not checked yetcites Balleine & O’Doherty, 2010
- Not checked yetcites Dickinson, 1985
- Not checked yetcites Dolan & Dayan, 2013
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Here, we have shown computationally that such value-based mechanisms are not strictly necessary, and that a value-free mechanism can account for the major behavioral phenomena that define habits.”
§ Habits without Values > Discussion
Load-bearingStrength:boosted· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The idea that separate goal-directed and habitual controllers exist in the brain, supported by distinct neural circuits, is strongly supported by lesion data from both humans and other animals.”
§ Habits without Values > Discussion > Tensions in Neuroscientific Data
Strength:boosted· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The finding that VI schedules result in rapid habit formation has been widely replicated and represents a key element of the experimental toolkit for the study of habits (Gremel & Costa, 2013; Tanaka, Balleine, & O’Doherty, 2008; Yin & Knowlton, 2006).”
§ Habits without Values > Results > Effects of Reinforcement Schedule on Habit Formation
What the sources say
- Not checked yetcites Gremel & Costa, 2013
- Not checked yetcites Tanaka, Balleine, & O’Doherty, 2008
- Not checked yetcites Yin & Knowlton, 2006
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The first of these features is that habits form slowly over time and often depend on behaviors that are initially taken under goal-directed control.”
§ Habits without Values > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“We first test whether our model can capture a central finding from research on habits: that extensive training in a static environment (overtraining) can render behavior inflexible in the face of changes to that environment.”
§ Habits without Values > Results > Effects of Training Duration on Behavioral Flexibility
What the sources say
- Not checked yetcites Dickinson, 1998
- Not checked yetcites Adams, 1982
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This example not only begins to validate the predictive abilities of our particular model, but also highlights the importance of a value-free habitual controller more generally in explaining habit-like behaviors that cannot otherwise be accounted for by a model-free RL-based algorithm alone.”
§ Habits without Values > Results > Perseverative Behavior in Sequential Choice Tasks
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“relative to control rats, rats with lesions to DMS lever-press at a lower rate and are less able to decrease their press rate when reinforcers are omitted or devalued (Yin et al., 2005)”
§ Habits without Values > Results > Effects of Striatal Lesions on Habit Formation and Behavioral Flexibility
What the sources say
- Not checked yetcites Yin et al., 2005
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“rats with lesions to DLS lever-press at a similar rate to controls and are more successful than controls at adapting their press rate to omitted or devalued reinforcement (Yin, Knowlton, & Balleine, 2004, 2006).”
§ Habits without Values > Results > Effects of Striatal Lesions on Habit Formation and Behavioral Flexibility
What the sources say
- Not checked yetcites Yin, Knowlton, & Balleine, 2004, 2006
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This computational account in which habits are understood as value-free stimulus-response associations therefore provides a closer match to classic psychological theories of habits, an account for classic behavioral data on habit formation, and a novel framework for understanding additional behavioral phenomena.”
§ Habits without Values > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearIt may state two separate claims as one.
In the paper
“In one recent example, rats performing a dynamic two-armed bandit task exhibited behavioral patterns consistent with both reinforcement-seeking (i.e., being more likely to select a recently reinforced action), as well as with choice perseveration (i.e., being more likely to select a recently chosen action).”
§ Habits without Values > Results > Perseverative Behavior in Sequential Choice Tasks
What the sources say
- Not checked yetcites Miller et al., in prep
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“The extra computational complexity associated with the goal-directed system was a source of uncertainty, leading the arbiter to favor the habitual system in well-learned environments.”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites Daw et al., 2005
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“This arbiter is similar to mechanisms found in computational accounts mapping habitual/goal-directed control onto model-free/model-based reinforcement learning.”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites Daw et al., 2005
- Not checked yetcites S. W. Lee et al., 2014
- Not checked yetcites Keramati et al., 2011
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“A large and influential body of computational work is built on the assumption that habitual control arises from model-free reinforcement learning algorithms (Dolan & Dayan, 2013; O’Doherty, Lee, & McNamee, 2015).”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites Dolan & Dayan, 2013
- Not checked yetcites O’Doherty, Lee, & McNamee, 2015
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Subsequent accounts have developed arbiters that consider proxies for this uncertainty (S. W. Lee et al., 2014), or other possible advantages of habitual over goal-directed control, such as response time (Keramati et al., 2011).”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites S. W. Lee et al., 2014
- Not checked yetcites Keramati et al., 2011
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In rodents, goal-directed behavior results in greater activity in OFC and DMS, while habitual behavior results in greater activity in DLS (Gremel & Costa, 2013).”
§ Habits without Values > Discussion > Tensions in Neuroscientific Data
What the sources say
- Not checked yetcites Gremel & Costa, 2013
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the value-free mechanisms we have proposed for habits by no means preclude a role for value or motivational state (e.g., hunger or satiety) in habit learning.”
§ Habits without Values > Discussion > Implications
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: In the model, control of behavior is allocated on each trial to either the habitual or the goal-directed system by an arbiter.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These may modulate the strengthening of habit associations either directly (e.g., through a feedback mechanism that influences the S-R association) or indirectly (e.g., by influencing the vigor of an action, which in turn results in greater associative strengths).”
§ Habits without Values > Discussion > Implications
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- The authors state this as an assumption: In the model, control of behavior is allocated on each trial to either the habitual or the goal-directed system by an arbiter.
None of the sources cited for this claim have been checked yet.
In the paper
“In human subjects, goal-directed value signals during outcome devaluation have been identified in vmPFC (Valentin, Dickinson, & O’Doherty, 2007), and similar signals during contingency degradation have been identified both in vmPFC and in the caudate nucleus, a homolog of DMS (Tanaka et al., 2008).”
§ Habits without Values > Discussion > Tensions in Neuroscientific Data
What the sources say
- Not checked yetcites Valentin, Dickinson, & O’Doherty, 2007
- Not checked yetcites Tanaka et al., 2008
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our proposed framework re-maps habits to the perseveration kernel and provides an account of how that kernel might plausibly operate in tandem with a goal-directed controller so as to account for behaviors that have previously been described by RL models.”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These data indicate that dopamine neurons are unlikely to play a role in a uniquely model-free control system, but instead have access to model-based information and play a role in model-based control.”
§ Habits without Values > Discussion > Tensions in Neuroscientific Data
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In effect, we are showing that the model-free component of some of these previous formalizations might not be necessary and that an elaboration of this perseveration kernel actually serves as a better model of habits.”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In sum, considerable evidence supports the idea that anatomically separate goal-directed and habitual controllers exist in the brain, and that either controller can be responsible for a given action.”
§ Habits without Values > Discussion > Tensions in Neuroscientific Data
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“habits are promoted in situations where the contingencies between actions and outcomes are weak (Dickinson, 1985).”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites Dickinson, 1985
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We first propose severing the tie between habits and model-free RL, and instead defining a category of behaviors that are “value-free” and therefore distinct from either type of RL computation.”
§ Habits without Values > Discussion > A Proposed Realignment
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
In the paper
“In the initial formalization of this idea (Daw et al., 2005), each system determined the uncertainty in its value estimates, and the arbiter selected the system with less overall uncertainty.”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites Daw et al., 2005
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“In addition to utilizing different mechanisms for value-based control and for arbitrating between the value-based and value-free controllers, our model also differs from these corticostriatal models in the level of analysis at which it is described. While this limits the level of detail with which our model can engage neurobiological data, it greatly facilitates engagement with a wide variety of”
§ Habits without Values > Discussion > Relationship to Previous Computational Models
What the sources say
- Not checked yetcites Frank, 2015
- Not checked yetcites Frank & Badre, 2015
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Beyond providing a potentially more parsimonious account of previous behavioral results”
§ Habits without Values > Conclusions
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: In the model, control of behavior is allocated on each trial to either the habitual or the goal-directed system by an arbiter.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“such a realignment may offer a better account of extant neural findings, including the fact that structures associated with model-free and model-based computations (i.e., value-based computations) tend to overlap, whereas lesion/inactivation studies have revealed clear dissociations between structures associated with goal-directed behavior versus (potentially value-free) habits.”
§ Habits without Values > Conclusions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearIt may state two separate claims as one.
- The authors state this as an assumption: In the model, control of behavior is allocated on each trial to either the habitual or the goal-directed system by an arbiter.
The paper relies on this without support.
The simulated task environments capture the essential features of the corresponding animal learning experiments, so that the model's behavior in them can be compared to animal behavior.
“To illustrate the behavior of the model and the dynamics of its various internal variables, we simulated behavior in a probabilistic reversal learning task (Fig 3).”
The paper relies on this without support.
The model's goal-directed controller corresponds to the dorsomedial striatum (DMS) and its habitual controller corresponds to the dorsolateral striatum (DLS).
“We lesioned the goal-directed controller (DMS) or habitual controller (DLS) in our model”
“Lesions to our model’s goal-directed and habitual controllers thus reproduce behavioral patterns typical of DMS and DLS lesioned rats”
The authors state this as an assumption.
In the model, control of behavior is allocated on each trial to either the habitual or the goal-directed system by an arbiter.
“In our model, control of behavior is allocated on each trial to either the habitual or the goal-directed system by an arbiter.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Supporting this, computational studies have proposed two parallel forms of reinforcement learning, model-based and model-free (or value-free) learning corresponding to goal-directed and habitual strategies, respectively, that develop concurrently during behavior acquisition2,41,42.”
From Dissociable roles of prefrontal plasticity in decision-making strategy and execution of habitual behavior (2026) · cites this paper as [42]
Experimental — accuracy not yet measuredIn the citing paper
“With overtraining, the habitual system simplifies decision-making by shifting to a stimulus-response mode of behavior51, making decisions potentially value-less and inflexible but less cognitively demanding52.”
From Revealing abrupt transitions from goal-directed to habitual behavior (2026) · cites this paper as [52]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Training reinforcement probability differs between Methods and Results
MajorThe Methods specify a 25% pellet probability during omission training, and say devaluation training was identical. The Results describe a 50% pellet probability for both training phases.
“In the first phase, performance of one action (Press Lever) resulted in a reinforcer of one type (Pellet) 25% of the time”
“The agent was initially trained in an environment where performance of Action A (“press lever”) was followed by a reinforcer of one type (“food pellet”) 50% of the time”
Habit-weight update uses a different basis function
MinorThe habit-strength function uses basis functions γi, so its derivative with respect to weight ci is γi. The stated weight-update equation instead substitutes χi, which is not the defined habit basis function.
“=ct(i)+αH⋅∑a′((δa′,at−H(a′))⋅χi(a′))”
“11Ht(a)=∑i=130ct(i)⋅γi(a)γi(a)=e−(a−5i)22⋅52”
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
James, W. (1891) · Personality Culture Society · doi:10.1037/11059-000 · Cited by 25,876
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256
The principles of psychology, Vol I.
James, W. (1890) · doi:10.1037/10538-000 · Cited by 4,514
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
The role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402
How are habits formed: modelling habit formation in the real world
Lally, P., van Jaarsveld, C. H. M., Potts, H. W. W., & Wardle, J. (2010) · European Journal of Social Psychology · doi:10.1002/ejsp.674 · Cited by 2,036
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
Flagged first, then by citations
1 of the 136 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Binding and Retrieval in Action Control (BRAC)
Frings, C., Hommel, B., Koch, I., Rothermund, K., Dignath, D., Giesen, C. G., Kiesel, A., Kunde, W., Mayr, S., Moeller, B., et al. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.02.004 · Cited by 379
Motivation and cognitive control in depression
Grahek, I., Shenhav, A., Musslick, S., Krebs, R. M., & Koster, E. H. W. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.04.011 · Cited by 389
Self-citationResolving uncertainty in a social world
FeldmanHall, O., & Shenhav, A. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0590-x · Cited by 298
Self-citationThe neural and computational systems of social learning
Olsson, A., Knapska, E., & Lindström, B. (2020) · Nature reviews. Neuroscience · doi:10.1038/s41583-020-0276-4 · Cited by 285
Habitual Behavior Is Goal-Driven
Kruglanski, A. W., & Szumowska, E. (2020) · Perspectives on Psychological Science · doi:10.1177/1745691620917676 · Cited by 231
Pro-environmental habits: An underexplored research agenda in sustainability science
Linder, N., Giusti, M., Samuelsson, K., & Barthel, S. (2021) · AMBIO · doi:10.1007/s13280-021-01619-6 · Cited by 210
Humans primarily use model-based inference in the two-stage task
Feher da Silva, C., & Hare, T. A. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0905-y · Cited by 153
Habits and Goals in Human Behavior: Separate but Interacting Systems
Wood, W., Mazar, A., & Neal, D. T. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691621994226 · Cited by 173
Beyond dichotomies in reinforcement learning
Collins, A. G. E., & Cockburn, J. (2020) · Nature reviews. Neuroscience · doi:10.1038/s41583-020-0355-6 · Cited by 147
The computational roots of positivity and confirmation biases in reinforcement learning
Palminteri, S., & Lebreton, M. P. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.04.005 · Cited by 134
Ho, M. K., Saxe, R. R., & Cushman, F. A. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.08.003 · Cited by 144
The case against economic values in the orbitofrontal cortex (or anywhere else in the brain).
Hayden, B. Y., & Niv, Y. (2021) · Behavioral Neuroscience · doi:10.1037/bne0000448 · Cited by 127
Sharpe, M. J., Stalnaker, T. A., Schuck, N. W., Killcross, S., Schoenbaum, G. M., & Niv, Y. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-010418-102824 · Cited by 117
Advances in modeling learning and decision-making in neuroscience
Collins, A. G. E., & Shenhav, A. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01126-y · Cited by 113
Self-citationTsutsui‐Kimura, I., Matsumoto, H., Akiti, K., Yamada, M. M., Uchida, N., & Watabe‐Uchida, M. (2020) · eLife · doi:10.7554/elife.62390 · Cited by 92
Origin of perseveration in the trade-off between reward and complexity
Gershman, S. J. (2020) · Cognition · doi:10.1016/j.cognition.2020.104394 · Cited by 105
The relationship between habits and motor skills in humans
Du, Y., Krakauer, J. W., & Haith, A. M. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.02.002 · Cited by 101
Perez, O. D., & Dickinson, A. (2020) · Psychological Review · doi:10.1037/rev0000201 · Cited by 98
Hierarchical Action Control: Adaptive Collaboration Between Actions and Habits
Balleine, B. W., & Dezfouli, A. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02735 · Cited by 98
Dopaminergic action prediction errors serve as a value-free teaching signal
Greenstreet, F., Vergara, H. M., Johansson, Y., Pati, S., Schwarz, L., Lenzi, S. C., Geerts, J. P., Wisdom, M., Gubanova, A., Rollik, L. B., et al. (2025) · Nature · doi:10.1038/s41586-025-09008-9 · Cited by 94
Time pressure changes how people explore and respond to uncertainty
Wu, C. M., Schulz, E., Pleskac, T. J., & Speekenbrink, M. (2022) · Scientific Reports · doi:10.1038/s41598-022-07901-1 · Cited by 103
Positive affect and behavior change
Shiota, M. N., Papies, E. K., Preston, S. D., & Sauter, D. A. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.04.022 · Cited by 90
Dopamine role in learning and action inference
Bogacz, R. (2020) · eLife · doi:10.7554/elife.53262 · Cited by 96
Linear reinforcement learning in planning, grid fields, and cognitive control
Piray, P., & Daw, N. D. (2021) · Nature Communications · doi:10.1038/s41467-021-25123-3 · Cited by 90
Context-dependent outcome encoding in human reinforcement learning
Palminteri, S., & Lebreton, M. P. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.06.006 · Cited by 87
The role of the lateral orbitofrontal cortex in creating cognitive maps
Costa, K. M., Scholz, R., Lloyd, K., Moreno‐Castilla, P., Gardner, M., Dayan, P., & Schoenbaum, G. M. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01216-0 · Cited by 78
Gardner, B., Rebar, A. L., de Wit, S., & Lally, P. J. (2024) · Social and Personality Psychology Compass · doi:10.1111/spc3.12975 · Cited by 93
van Elzelingen, W., Warnaar, P., de Matos, J. M. S., Bastet, W., Jonkman, R., Smulders, D., Goedhoop, J. N., Denys, D. A. J. P., Arbab, T., & Willuhn, I. (2022) · Current Biology · doi:10.1016/j.cub.2021.12.027 · Cited by 63
Value representations in the rodent orbitofrontal cortex drive learning, not choice
Miller, K. J., Botvinick, M., & Brody, C. D. (2022) · eLife · doi:10.7554/elife.64575 · Cited by 72
Self-citationFintz, M., Osadchy, M., & Hertz, U. (2022) · Scientific Reports · doi:10.1038/s41598-022-08863-0 · Cited by 68
Reward prediction error does not explain movement selectivity in DMS-projecting dopamine neurons
Lee, R. S., Mattar, M. G., Parker, N. F., Witten, I. B., & Daw, N. D. (2019) · eLife · doi:10.7554/elife.42992 · Cited by 59
A Normative Account of Confirmation Bias During Reinforcement Learning
Lefebvre, G., Summerfield, C., & Bogacz, R. (2021) · Neural Computation · doi:10.1162/neco_a_01455 · Cited by 58
Making habits measurable beyond what they are not: A focus on associative dual-process models
Watson, P., O’Callaghan, C., Perkes, I. E., Bradfield, L. A., & Turner, K. M. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104869 · Cited by 67
Reinforcement learning and its connections with neuroscience and psychology
Subramanian, A., Chitlangia, S., & Baths, V. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.10.003 · Cited by 66
The Law of Recency: An Episodic Stimulus-Response Retrieval Account of Habit Acquisition
Giesen, C. G., Schmidt, J. R., & Rothermund, K. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02927 · Cited by 60
Bavard, S., Rustichini, A., & Palminteri, S. (2021) · Science Advances · doi:10.1126/sciadv.abe0340 · Cited by 53
Frömer, R., & Shenhav, A. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.12.006 · Cited by 57
Self-citationPolicy compression: An information bottleneck in action selection
Lai, L., & Gershman, S. (2021) · The psychology of learning and motivation · doi:10.1016/bs.plm.2021.02.004 · Cited by 77
Gardner, B., & Lally, P. J. (2022) · Health Psychology Review · doi:10.1080/17437199.2022.2105249 · Cited by 64
Zhang, C., Lakens, D., & IJsselsteijn, W. A. (2021) · Journal of Medical Internet Research · doi:10.2196/17127 · Cited by 60
Interfacing behavioral and neural circuit models for habit formation
Lerner, T. N. (2020) · Journal of Neuroscience Research · doi:10.1002/jnr.24581 · Cited by 50
Cognitive Control as a Multivariate Optimization Problem
Ritz, H., Leng, X., & Shenhav, A. (2022) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01822 · Cited by 57
Self-citationGenerating Options and Choosing Between Them Depend on Distinct Forms of Value Representation
Morris, A., Phillips, J., Huang, K., & Cushman, F. A. (2021) · Psychological Science · doi:10.1177/09567976211005702 · Cited by 54
Caching mechanisms for habit formation in Active Inference
Maisto, D., Friston, K. J., & Pezzulo, G. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.05.083 · Cited by 49
Value-free reinforcement learning: policy optimization as a minimal model of operant behavior
Bennett, D., Niv, Y., & Langdon, A. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.04.020 · Cited by 48
Smith, A. C. W., Jonkman, S., DiFeliceantonio, A. G., O’Connor, R. M., Ghoshal, S., Romano, M. F., Everitt, B. J., & Kenny, P. J. (2021) · Nature Communications · doi:10.1038/s41467-021-25460-3 · Cited by 46
Linder, N., Rosenthal, S., Sörqvist, P., & Barthel, S. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.699410 · Cited by 56
Choice-confirmation bias and gradual perseveration in human reinforcement learning.
Palminteri, S. (2022) · Behavioral Neuroscience · doi:10.1037/bne0000541 · Cited by 45
Studying Human Habits in Societal Context: Examining Support for a Basic Stimulus–Response Mechanism
Marien, H., Ruud Custers, A., & Aarts, H. (2019) · Current Directions in Psychological Science · doi:10.1177/0963721419868211 · Cited by 39
The Willpower Paradox: Possible and Impossible Conceptions of Self-Control
Goschke, T., & Job, V. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916221146158 · Cited by 53
Realigning Models of Habitual and Goal-Directed Decision-Making
Miller, K. J., Ludvig, E. A., Pezzulo, G., & Shenhav, A. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-812098-9.00018-8 · Cited by 46
Self-citationThe affective gradient hypothesis: An affect-centered account of motivated behavior
Shenhav, A. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.08.003 · Cited by 49
Self-citationLeveraging cognitive neuroscience for making and breaking real-world habits
Buabang, E. K., Donegan, K. R., Rafei, P., & Gillan, C. M. (2025) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.10.006 · Cited by 34
The learning of prospective and retrospective cognitive maps within neural circuits
Namboodiri, V. M. K., & Stuber, G. D. (2021) · Neuron · doi:10.1016/j.neuron.2021.09.034 · Cited by 32
Goal-Directed and Habitual Control in Human Substance Use: State of the Art and Future Directions
Doñamayor, N., Ebrahimi, C., Arndt, V. A., Weiß, F., Schlagenhauf, F., & Endraß, T. (2022) · Neuropsychobiology · doi:10.1159/000527663 · Cited by 40
A mismatch between striatal cholinergic pauses and dopaminergic reward prediction errors
Duhne, M., Mohebi, A., Kim, K., Pelattini, L., & Berke, J. D. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2410828121 · Cited by 37
Marzuki, A. A., Tomić, I., Ip, S. H. Y., Gottwald, J., Kanen, J. W., Kaser, M., Sule, A., Conway-Morris, A., Sahakian, B. J., & Robbins, T. W. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.36195 · Cited by 32
An fMRI meta-analysis of the role of the striatum in everyday-life vs laboratory-developed habits
Guida, P., Michiels, M., Redgrave, P., Luque, D., & Obeso, I. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104826 · Cited by 32
Wu, W., Hu, L., Chen, Y., Cao, F., Ding, S., Wu, T., & Xu, J. (2022) · BMC Oral Health · doi:10.1186/s12903-022-02219-w · Cited by 32
Characterizing Human Habits in the Lab
Nebe, S., Kretzschmar, A., Brandt, M. C., & Tobler, P. N. (2024) · Collabra Psychology · doi:10.1525/collabra.92949 · Cited by 37
Hackel, L. M., Berg, J. J., Lindström, B., & Amodio, D. M. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02592 · Cited by 31
Pro-environmental messages have more effect when they come from less familiar brands
Herédia-Colaço, V. (2022) · Journal of Product & Brand Management · doi:10.1108/jpbm-12-2021-3782 · Cited by 30
Parvizi-Wayne, D., Sandved-Smith, L., Pitliya, R. J., Limanowski, J., Tufft, M. R. A., & Friston, K. J. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1354719 · Cited by 37
Binary Theorizing Does Not Account for Action Control
Hommel, B. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02542 · Cited by 35
Balancing control: A Bayesian interpretation of habitual and goal-directed behavior
Schwöbel, S., Marković, D., Smolka, M. N., & Kiebel, S. J. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102472 · Cited by 31
Moneta, N., Grossman, S., & Schuck, N. W. (2024) · Trends in Neurosciences · doi:10.1016/j.tins.2024.10.005 · Cited by 37
Luo, Q., Kanen, J. W., Bari, A., Skandali, N., Langley, C., Knudsen, G. M., Alsiö, J., Phillips, B. U., Sahakian, B. J., Cardinal, R. N., et al. (2023) · Neuropsychopharmacology · doi:10.1038/s41386-023-01762-6 · Cited by 26
Herz, D. M., Frank, M. J., Tan, H., & Groppa, S. (2024) · Brain · doi:10.1093/brain/awae184 · Cited by 31
Mixing memory and desire: How memory reactivation supports deliberative decision‐making
Wang, S., Feng, S. F., & Bornstein, A. M. (2021) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1581 · Cited by 28
Reinforcer predictability and stimulus salience promote discriminated habit learning.
Thrailkill, E. A., Michaud, N. L., & Bouton, M. E. (2021) · Journal of Experimental Psychology Animal Learning and Cognition · doi:10.1037/xan0000285 · Cited by 30
Group value learned through interactions with members: A reinforcement learning account
Hackel, L. M., Kogon, D., Amodio, D. M., & Wood, W. (2021) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2021.104267 · Cited by 29
Value Representations in the Rodent Orbitofrontal Cortex Drive Learning, not ChoicePreprint
Miller, K. J., Botvinick, M., & Brody, C. D. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/245720 · Cited by 27
Self-citationInvasive Computational Psychiatry
Sáez, I., & Gu, X. (2022) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.09.032 · Cited by 27
Model-Free RL or Action Sequences?
Morris, A., & Cushman, F. A. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02892 · Cited by 28
Miller, K. J., Botvinick, M., & Brody, C. D. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/461129 · Cited by 25
Self-citationTheory-based habit modeling for enhancing behavior prediction in behavior change support systems
Zhang, C., Vanschoren, J., van Wissen, A., Lakens, D., de Ruyter, B., & IJsselsteijn, W. A. (2022) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-022-09326-x · Cited by 24
Akam, T., Rodrigues-Vaz, I., Marcelo, I., Zhang, X., Pereira, M., Oliveira, R. F., Dayan, P., & Costa, R. M. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/126292 · Cited by 25
Eckstein, M. K., Summerfield, C., Daw, N. D., & Miller, K. J. (2023) · doi:10.1101/2023.05.17.541226 · Cited by 18
Self-citationThe consequences of AI training on human decision-making
Treiman, L. S., Ho, C.-J., & Kool, W. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2408731121 · Cited by 24
Schmidt, J. R. (2021) · L’Année psychologique · doi:10.3917/anpsy1.212.0077 · Cited by 24
Tanner, C., Schmocker, D., Katsarov, J. W., & Christen, M. (2021) · Computers & Education · doi:10.1016/j.compedu.2021.104381 · Cited by 27
Frith, C., & Frith, U. (2023) · The MIT Press eBooks · doi:10.7551/mitpress/10400.001.0001 · Cited by 22
Associative Learning from Replayed ExperiencePreprint
Ludvig, E. A., Mirian, M. S., Kehoe, E. J., & Sutton, R. S. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/100800 · Cited by 20
Self-citationPiray, P., & Daw, N. D. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/856849 · Cited by 22
Ferry, L., Funnell, W., & Oldroyd, D. (2023) · Accounting, Organizations and Society · doi:10.1016/j.aos.2023.101466 · Cited by 23
Neurobehavioral Precursors of Compulsive Cocaine Seeking in Dual Frontostriatal Circuits
Jones, J. A., Belin‐Rauscent, A., Jupp, B., Fouyssac, M., Sawiak, S. J., Zühlsdorff, K., Zhukovsky, P., Hebdon, L., Velázquez-Sánchez, C., Robbins, T. W., et al. (2023) · Biological Psychiatry Global Open Science · doi:10.1016/j.bpsgos.2023.06.001 · Cited by 18
Experiential values are underweighted in decisions involving symbolic options
Garcia, B., Lebreton, M. P., Bourgeois‐Gironde, S., & Palminteri, S. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01496-3 · Cited by 28
Toyama, A., Katahira, K., & Kunisato, Y. (2023) · Computational Brain & Behavior · doi:10.1007/s42113-023-00175-4 · Cited by 18
Testing models of context-dependent outcome encoding in reinforcement learning
Hayes, W. M., & Wedell, D. H. (2022) · Cognition · doi:10.1016/j.cognition.2022.105280 · Cited by 23
Representation of Anticipated Rewards and Punishments in the Human Brain
Ben‐Zion, Z., & Levy, I. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-022324-042614 · Cited by 21
Old Strategies, New Environments: Reinforcement Learning on Social Media
Turner, G., Ferguson, A. M., Katiyar, T., Palminteri, S., & Orben, A. C. (2024) · Biological Psychiatry · doi:10.1016/j.biopsych.2024.12.012 · Cited by 35
Gamification of Behavior Change: Mathematical Principle and Proof-of-Concept Study
Lieder, F., Chen, P.-Z., Prentice, M. S., Amo, V., & Tošić, M. (2023) · JMIR Serious Games · doi:10.2196/43078 · Cited by 24
Environmental habitus of chefs and food waste management in hospitality operations
Bhaskara, G. I., Filimonau, V., Kusumawardhana, I., Putra, I. N. D., Yanthy, P. S., & Hakim, I. N. (2024) · Journal of Sustainable Tourism · doi:10.1080/09669582.2024.2408008 · Cited by 18
Womelsdorf, T., Watson, M. R., & Tiesinga, P. H. E. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01780 · Cited by 20
Theory-driven computational models of drug addiction in humans: Fruitful or futile?
Lim, T. V., & Ersche, K. D. (2023) · Addiction Neuroscience · doi:10.1016/j.addicn.2023.100066 · Cited by 16
Computational approaches to habits in a model-free world
Pauli, W. M., Cockburn, J., Pool, E. R., Perez, O. D., & O’Doherty, J. P. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.12.001 · Cited by 18
de Oliveira, A. R., Reimer, A. E., Simandl, G. J., Nagrale, S. S., & Widge, A. S. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01448-x · Cited by 14
Zhang, C., Adriaanse, M. A., Potgieter, R., Tummers, L. G., de Wit, J. B. F., Broersen, J., de Bruin, M., & Aarts, H. (2022) · BMC Public Health · doi:10.1186/s12889-022-13977-1 · Cited by 15
Zühlsdorff, K., López‐Cruz, L., Dutcher, E. G., Jones, J. A., Pama, C., Sawiak, S. J., Khan, S. A., Milton, A. L., Robbins, T. W., Bullmore, E. T., et al. (2022) · Neurobiology of Stress · doi:10.1016/j.ynstr.2022.100507 · Cited by 15
Representational Exchange in Human Social Learning
Wu, C. M., Vélez, N., & Cushman, F. A. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009026949.009 · Cited by 21
Metadata
- Authors
- Miller, K. J., Shenhav, A., & Ludvig, E. A.
- Affiliations
- Princeton University (US)
- Venue
- Psychological Review
- Published
- 2019-03
- Volume
- 126
- Issue
- 2
- Pages
- 292-311
- ISSN
- 0033-295X (ISSN-L)
- 1939-1471
- PMC
- PMC6548181
- OpenAlex
- W2949941589
- Cited by
- 280
- References
- 136
- Citations by year
- 2026: 32 · 2025: 38 · 2024: 52 · 2023: 29 · 2022: 41 · 2021: 41 · 2020: 24 · 2019: 14
- FWCI
- 17.32
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Psychological Review (publishedVersion, other-oa)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Warwick Research Archive Portal (University of Warwick) (submittedVersion)
- Funders
- National Institutes of Health (T-32 MH065214)
- Princeton University
- Sponsor name not included
- National Institute of General Medical Sciences (P20GM103645)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Psychology › Applied Psychology › Behavioral Health and Interventions
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- habit, stimulus-response learning, model-free reinforcement learning, outcome devaluation, contingency degradation, reinforcement schedules, perseveration, instrumental behavior, habit formation
- MeSH terms
- Animals, Behavior, Animal, Decision Making, Goals, Habits, Humans, Models, Psychological, Reinforcement, Psychology, Prefrontal Cortex
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
