The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Reinforcement-Based Decision Making in Corticostriatal Circuits: Mutual Constraints by Neurocomputational and Diffusion Models
Abstract
In this letter, we examine the computational mechanisms of reinforce-ment-based decision making. We bridge the gap across multiple levels of analysis, from neural models of corticostriatal circuits—the basal ganglia (BG) model (Frank, 2005 , 2006 ) to simpler but mathematically tractable diffusion models of two-choice decision making. Specifically, we generated simulated data from the BG model and fit the diffusion model (Ratcliff, 1978 ) to it. The standard diffusion model fits underestimated response times under conditions of high response and reinforcement conflict. Follow-up fits showed good fits to the data both by increasing nondecision time and by raising decision thresholds as a function of conflict and by allowing this threshold to collapse with time. This profile captures the role and dynamics of the subthalamic nucleus in BG circuitry, and as such, parametric modulations of projection strengths from this nucleus were associated with parametric increases in decision boundary and its modulation by conflict. We then present data from a human reinforcement learning experiment involving decisions with low- and high-reinforcement conflict. Again, the standard model failed to fit the data, but we found that two variants similar to those that fit the BG model data fit the experimental data, thereby providing a convergence of theoretical accounts of complex interactive decision-making mechanisms consistent with available data. This work also demonstrates how to make modest modifications to diffusion models to summarize core computations of the BG model. The result is a better fit and understanding of reinforcement-based choice data than that which would have occurred with either model alone.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Effects of noise letters upon the identification of a target letter in a nonsearch task
Eriksen, B. A., & Eriksen, C. W. (1974) · Perception & Psychophysics · doi:10.3758/bf03203267 · Cited by 7,230
Ratcliff, R. (1978) · Psychological Review · doi:10.1037/0033-295x.85.2.59 · Cited by 4,513 · cited 1×
“The diffusion model (Ratcliff, 1978) is a model of the cognitive processes involved in making simple two-choice decisions (see Figure 2).”
Self-citationThe Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764
The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks
Ratcliff, R., & McKoon, G. (2008) · Neural Computation · doi:10.1162/neco.2008.12-06-420 · Cited by 3,059 · cited 3×
“…one accumulator or two accumulators) were successful in accounting for the qualitative patterns of results found in two-choice tasks (see Bogacz, Brown, Moehlis, Holmes, & Cohen, 2006; Ratcliff, 2006; Ratcliff & McKoon, 2008; Ratcliff, Thapar, Smith, & McKoon, 2005; Usher & McClelland, 2001).”
BackgroundMethodSelf-citationThe time course of perceptual choice: The leaky, competing accumulator model.
Usher, M., & McClelland, J. L. (2001) · Psychological Review · doi:10.1037/0033-295x.108.3.550 · Cited by 2,152 · cited 4×
“…one accumulator or two accumulators) were successful in accounting for the qualitative patterns of results found in two-choice tasks (see Bogacz, Brown, Moehlis, Holmes, & Cohen, 2006; Ratcliff, 2006; Ratcliff & McKoon, 2008; Ratcliff, Thapar, Smith, & McKoon, 2005; Usher & McClelland, 2001).”
BackgroundMethodInfluentialVisual fixations and the computation and comparison of value in simple choice
Krajbich, I., Armel, C., & Rangel, A. (2010) · Nature Neuroscience · doi:10.1038/nn.2635 · Cited by 1,408
Aron, A. R., Behrens, T. E., Smith, S., Frank, M. J., & Poldrack, R. A. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0519-07.2007 · Cited by 982
Self-citationRatcliff, R., & Tuerlinckx, F. (2002) · Psychonomic Bulletin & Review · doi:10.3758/bf03196302 · Cited by 838 · cited 6×
“…are multiplied by the number of observations in the condition to give observed ( O ) and expected ( E ) frequencies and summing over ( O − E ) 2 / E for all conditions gives a single chi square value to be minimized (see Ratcliff & Tuerlinckx, 2002, for a full description of the method).”
BackgroundMethodInfluentialSelf-citationHold your horses: a dynamic computational role for the subthalamic nucleus in decision making
Frank, M. J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.006 · Cited by 706
Self-citationDecision-making with multiple alternatives
Churchland, A. K., Kiani, R., & Shadlen, M. N. (2008) · Nature Neuroscience · doi:10.1038/nn.2123 · Cited by 671
Cortico–basal ganglia circuit mechanism for a decision threshold in reaction time tasks
Lo, C.-C., & Wang, X.-J. (2006) · Nature Neuroscience · doi:10.1038/nn1722 · Cited by 505
Jahfari, S., Waldorp, L., van den Wildenberg, W. P. M., Scholte, H. S., Ridderinkhof, K. R., & Forstmann, B. U. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.5253-10.2011 · Cited by 308
A neurobiological theory of automaticity in perceptual categorization
Ashby, F. G., Ennis, J. M., & Spiering, B. J. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.632 · Cited by 286
Ratcliff, R., Hasegawa, Y. T., Hasegawa, R. P., Smith, P. L., & Segraves, M. A. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00393.2006 · Cited by 236
Self-citationStochastic models of decisions about motion direction: Behavior and physiology
Ditterich, J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.05.042 · Cited by 233
Dissociating early and late error signals in perceptual recognition
Wheeler, M. E., Petersen, S. E., Nelson, S. M., Ploran, E. J., & Velanova, K. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2008.20155 · Cited by 30
Cited by
Diffusion decision model: Current issues and history
Ratcliff, R., Smith, P. L., Brown, S. D., & McKoon, G. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.01.007 · Cited by 1,630
“The conflict conditions were modeled with collapsing decision boundaries that accounted for the shifts in RT distributions [69].”
BackgroundMethodSelf-citationNeuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
HDDM: Hierarchical Bayesian estimation of the Drift-Diffusion Model in Python
Wiecki, T. V., Sofer, I., & Frank, M. J. (2013) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2013.00014 · Cited by 1,047
Self-citationMetacognition in human decision-making: confidence and error monitoring
Yeung, N., & Summerfield, C. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2011.0416 · Cited by 667
Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold
Cavanagh, J. F., Wiecki, T. V., Cohen, M. X., Figueroa, C. M., Samanta, J., Sherman, S. J., & Frank, M. J. (2011) · Nature Neuroscience · doi:10.1038/nn.2925 · Cited by 645
Self-citation
Metadata
- Authors
- Ratcliff, R., & Frank, M. J.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2012-05
- Volume
- 24
- Issue
- 5
- Pages
- 1186-1229
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W1997904803
- Cited by
- 207
- References
- 16
- FWCI
- 8.60
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.