The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
From recurrent choice to skill learning: A reinforcement-learning model.
Open accessAbstract
The authors propose a reinforcement-learning mechanism as a model for recurrent choice and extend it to account for skill learning. The model was inspired by recent research in neurophysiological studies of the basal ganglia and provides an integrated explanation of recurrent choice behavior and skill learning. The behavior includes effects of differential probabilities, magnitudes, variabilities, and delay of reinforcement. The model can also produce the violation of independence, preference reversals, and the goal gradient of reinforcement in maze learning. An experiment was conducted to study learning of action sequences in a multistep task. The fit of the model to the data demonstrated its ability to account for complex skill learning. The advantages of incorporating the mechanism into a larger cognitive architecture are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
A learning algorithm for Boltzmann machines
Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_7 · Cited by 3,402
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 1×
“For example, Erev and Barron showed that their model described the learning trends in Myers et al. (1965) and some of the similar effects such as probability learning that we presented in Part 1 of our section Testing the Mechanism Against Empirical Data .I n addition, there are reinforcement-learning models that show…”
BackgroundThe neural basis of error detection: Conflict monitoring and the error-related negativity
Yeung, N., Botvinick, M. M., & Cohen, J. D. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.931 · Cited by 1,924
Toward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450 · cited 3×
“One of the most common policies in reinforcement learning is called the " soft-max " function (see, e.g., Sutton & Barto, 1981), which, coincidentally, is also a part of the ACT–”
BackgroundInfluentialFunctional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404
Schultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Responses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Independent Coding of Reward Magnitude and Valence in the Human Brain
Yeung, N., & Sanfey, A. G. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.4537-03.2004 · Cited by 933 · cited 3×
“This is consistent with results from the measurement of event-related brain potential in a similar gambling task (Yeung & Sanfey, 2004), in which participants were shown the outcome of the alternative they chose and the outcome of the alternative they did not choose.”
BackgroundResultThe goal-gradient hypothesis and maze learning.
Hull, C. L. (1932) · Psychological Review · doi:10.1037/h0072640 · Cited by 593
Importance of unpredictability for reward responses in primate dopamine neurons
Mirenowicz, J., & Schultz, W. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.72.2.1024 · Cited by 569
Activity in human ventral striatum locked to errors of reward prediction
Pagnoni, G., Zink, C. F., Montague, P. R., & Berns, G. S. (2002) · Nature Neuroscience · doi:10.1038/nn802 · Cited by 451 · cited 1×
“Inversely, when a fully predicted reward fails to occur, DA neurons are depressed in their activity at exactly the time when the reward would have occurred ( Pagnoni, Zink, Montague, & Berns, 2002; Schultz, Apicella, & Ljungberg, 1993).”
BackgroundDorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations
DELGADO, M. R., LOCKE, H. M., STENGER, V. A., & FIEZ, J. A. (2003) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.3.1.27 · Cited by 431
Gray, W. D., Sims, C. R., Fu, W.-T., & Schoelles, M. J. (2006) · Psychological Review · doi:10.1037/0033-295x.113.3.461 · Cited by 360
Self-citationHuman Symbol Manipulation Within an Integrated Cognitive Architecture
Anderson, J. R. (2005) · Cognitive Science · doi:10.1207/s15516709cog0000_22 · Cited by 253
Self-citationDopamine, uncertainty and TD learning
Niv, Y., Duff, M. O., & Dayan, P. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-6 · Cited by 153
Cited by
Mindful Judgment and Decision Making
Weber, E. U., & Johnson, E. J. (2009) · Annual Review of Psychology · doi:10.1146/annurev.psych.60.110707.163633 · Cited by 745
Gray, W. D., Sims, C. R., Fu, W.-T., & Schoelles, M. J. (2006) · Psychological Review · doi:10.1037/0033-295x.113.3.461 · Cited by 360
“The technique has also recently attracted the attention of the greater cognitive modeling community ( Fu & Anderson, 2004, 2006; Nason & Laird, 2004; Phillips & Noelle, 2004; Wu & Liu, 2004).”
MethodSelf-citationAn integrated model of cognitive control in task switching.
Altmann, E. M., & Gray, W. D. (2008) · Psychological Review · doi:10.1037/0033-295x.115.3.602 · Cited by 298
SNIF-ACT: A Cognitive Model of User Navigation on the World Wide Web
Fu, W.-T., & Pirolli, P. (2007) · Human–Computer Interaction · doi:10.1080/07370020701638806 · Cited by 276
Self-citationGoal-directed control and its antipodes
Dayan, P. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.004 · Cited by 93
Metadata
- Authors
- Fu, W.-T., & Anderson, J. R.
- Venue
- Journal of Experimental Psychology: General
- Published
- 2006
- Volume
- 135
- Issue
- 2
- Pages
- 184-206
- ISSN
- 0096-3445 (ISSN-L)
- 1939-2222
- OpenAlex
- W1964571600
- Cited by
- 167
- References
- 18
- FWCI
- 5.68
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.