The rat approximates an ideal detector of changes in rates of reward: Implications for the law of effect.
Abstract
Rats responded on 2 levers delivering brain stimulation reward on concurrent variable interval schedules. Following many successive sessions with unchanging relative rates of reward, subjects adjusted to an eventual change slowly and showed spontaneous reversions at the beginning of subsequent sessions. When changes in rates of reward occurred between and within every session, subjects adjusted to them about as rapidly as they could in principle do so, as shown by comparison to a Bayesian model of an ideal detector. This and other features of the adjustments to frequent changes imply that the behavioral effect of reinforcement depends on the subject's perception of incomes and changes in incomes rather than on the strengthening and weakening of behaviors in accord with their past effects or expected results. Models for the process by which perceived incomes determine stay durations and for the process that detects changes in rates are developed.
Cites (4)
- A neural substrate of prediction and rewardSchultz, Wolfram, Dayan, Peter, Montague, P. Read · 1997 · Science
- A framework for mesencephalic dopamine systems based on predictive Hebbian learningMontague, PR, Dayan, P, Sejnowski, TJ · 1996 · The Journal of Neuroscience
- Context, time, and memory retrieval in the interference paradigms of Pavlovian learningBouton, Mark E. · 1993 · Psychological Bulletin
- A MARKOV MODEL DESCRIPTION OF CHANGEOVER PROBABILITIES ON CONCURRENT VARIABLE‐INTERVAL SCHEDULES1Heyman, Gene M. · 1979 · Journal of the Experimental Analysis of Behavior
Cited by (19)
- An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performanceAston-Jones, Gary, Cohen, Jonathan D. · 2005 · Annual Review of Neuroscience
- Learning the value of information in an uncertain worldBehrens, Timothy E J, Woolrich, Mark W, Walton, Mark E, Rushworth, Matthew F S · 2007 · Nature Neuroscience
- Matching Behavior and the Representation of Value in the Parietal CortexSugrue, Leo P., Corrado, Greg S., Newsome, William T. · 2004 · Science
- Robust speech perception: Recognize the familiar, generalize to the similar, and adapt to the novel.Kleinschmidt, Dave F., Jaeger, T. Florian · 2015 · Psychological Review
- The learning curve: Implications of a quantitative analysisGallistel, Charles R., Fairhurst, Stephen, Balsam, Peter · 2004 · Proceedings of the National Academy of Sciences
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The rat approximates an ideal detector of changes in rates of reward: Implications for the law of effect. https://app.cochat.ai/papers/10.1037//0097-7403.27.4.354
- The paper
- Gallistel, C. R., Mark, T. A., King, A. P., & Latham, P. E. (2001). The rat approximates an ideal detector of changes in rates of reward: Implications for the law of effect. Journal of Experimental Psychology: Animal Behavior Processes. https://doi.org/10.1037//0097-7403.27.4.354
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.