The rat approximates an ideal detector of changes in rates of reward: Implications for the law of effect.

Gallistel, C. R., Mark, Terence A., King, Adam Philip, Latham, P. E.·2001·Journal of Experimental Psychology: Animal Behavior Processes

Abstract

Rats responded on 2 levers delivering brain stimulation reward on concurrent variable interval schedules. Following many successive sessions with unchanging relative rates of reward, subjects adjusted to an eventual change slowly and showed spontaneous reversions at the beginning of subsequent sessions. When changes in rates of reward occurred between and within every session, subjects adjusted to them about as rapidly as they could in principle do so, as shown by comparison to a Bayesian model of an ideal detector. This and other features of the adjustments to frequent changes imply that the behavioral effect of reinforcement depends on the subject's perception of incomes and changes in incomes rather than on the strengthening and weakening of behaviors in accord with their past effects or expected results. Models for the process by which perceived incomes determine stay durations and for the process that detects changes in rates are developed.

Cites (4)

  1. A neural substrate of prediction and reward
    Schultz, Wolfram, Dayan, Peter, Montague, P. Read · 1997 · Science
  2. A framework for mesencephalic dopamine systems based on predictive Hebbian learning
    Montague, PR, Dayan, P, Sejnowski, TJ · 1996 · The Journal of Neuroscience
  3. Context, time, and memory retrieval in the interference paradigms of Pavlovian learning
    Bouton, Mark E. · 1993 · Psychological Bulletin
  4. A MARKOV MODEL DESCRIPTION OF CHANGEOVER PROBABILITIES ON CONCURRENT VARIABLE‐INTERVAL SCHEDULES1
    Heyman, Gene M. · 1979 · Journal of the Experimental Analysis of Behavior

Show all 4 references

Cited by (19)

  1. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance
    Aston-Jones, Gary, Cohen, Jonathan D. · 2005 · Annual Review of Neuroscience
  2. Learning the value of information in an uncertain world
    Behrens, Timothy E J, Woolrich, Mark W, Walton, Mark E, Rushworth, Matthew F S · 2007 · Nature Neuroscience
  3. Matching Behavior and the Representation of Value in the Parietal Cortex
    Sugrue, Leo P., Corrado, Greg S., Newsome, William T. · 2004 · Science
  4. Robust speech perception: Recognize the familiar, generalize to the similar, and adapt to the novel.
    Kleinschmidt, Dave F., Jaeger, T. Florian · 2015 · Psychological Review
  5. The learning curve: Implications of a quantitative analysis
    Gallistel, Charles R., Fairhurst, Stephen, Balsam, Peter · 2004 · Proceedings of the National Academy of Sciences

See all 225

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The rat approximates an ideal detector of changes in rates of reward: Implications for the law of effect. https://app.cochat.ai/papers/10.1037//0097-7403.27.4.354
The paper
Gallistel, C. R., Mark, T. A., King, A. P., & Latham, P. E. (2001). The rat approximates an ideal detector of changes in rates of reward: Implications for the law of effect. Journal of Experimental Psychology: Animal Behavior Processes. https://doi.org/10.1037//0097-7403.27.4.354

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.