The DOI matches its OpenAlex record; 254 of the 254 references it cites were found in a registry or an index.
Mathematical principles of reinforcement
Abstract
Abstract Effective conditioning requires a correlation between the experimenter's definition of a response and an organism's, but an animal's perception of its behavior differs from ours. These experiments explore various definitions of the response, using the slopes of learning curves to infer which comes closest to the organism's definition. The resulting exponentially weighted moving average provides a model of memory that is used to ground a quantitative theory of reinforcement. The theory assumes that: incentives excite behavior and focus the excitement on responses that are contemporaneous in memory. The correlation between the organism's memory and the behavior measured by the experimenter is given by coupling coefficients, which are derived for various schedules of reinforcement. The coupling coefficients for simple schedules may be concatenated to predict the effects of complex schedules. The coefficients are inserted into a generic model of arousal and temporal constraint to predict response rates under any scheduling arrangement. The theory posits a response-indexed decay of memory, not a time-indexed one. It requires that incentives displace memory for the responses that occur before them, and may truncate the representation of the response that brings them about. As a contiguity-weighted correlation model, it bridges opposing views of the reinforcement process. By placing the short-term memory of behavior in so central a role, it provides a behavioral account of a key cognitive process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–254 of 254
Cellular investigations of behavioral reinforcement
Stein, L., & Belluzzi, J. D. (1989) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(89)80014-4 · Cited by 76
Bradshaw, C. M., Szabadi, E., & Bevan, P. (1978) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1978.29-447 · Cited by 76
Critical influence of particular experiences in the perception of letters, words, and phrases
Whittlesea, B. W. A., & Brooks, L. R. (1988) · Memory & Cognition · doi:10.3758/bf03214219 · Cited by 75
Marking in pigeons: The role of memory in delayed reinforcement.
Lieberman, D. A., Davidson, F. H., & Thomas, G. V. (1985) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.11.4.611 · Cited by 75
Permanent effects of punishment during extinction.
Boe, E. E., & Church, R. M. (1967) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0024632 · Cited by 74
Frequent reward eliminates differences in activity between hyperkinetic rats and controls
Sagvolden, T., Metzger, M. A., & Sagvolden, G. (1993) · Behavioral and Neural Biology · doi:10.1016/0163-1047(93)90986-r · Cited by 72
PREFERENCE FOR STARTING AND FINISHING BEHAVIOR PATTERNS
Shimp, C. P., Sabulsky, S. L., & Childers, L. J. (1989) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1989.52-341 · Cited by 71
Prediction of concurrent keypeck treadle-press responding from simple schedule performance
McSweeney, F. K. (1978) · Animal Learning & Behavior · doi:10.3758/bf03209642 · Cited by 70
EFFECTS OF REINFORCING PRESELECTED APPROXIMATIONS ON THE TOPOGRAPHY OF THE RAT'S BAR PRESS
Stokes, P. D., & Balsam, P. D. (1991) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1991.55-213 · Cited by 69
How much processing do nonattended stimuli receive? Apparently very little, but...
Eriksen, C. W., Webb, J. M., & Fournier, L. R. (1990) · Perception & Psychophysics · doi:10.3758/bf03208181 · Cited by 68
Wogar, M. A., Bradshaw, C. M., & Szabadi, E. (1991) · Psychopharmacology · doi:10.1007/bf02316873 · Cited by 68
VARIABLE‐RATIO SCHEDULES AS VARIABLE‐INTERVAL SCHEDULES WITH LINEAR FEEDBACK LOOPS
McDowell, J. J., & Wixted, J. T. (1986) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1986.46-315 · Cited by 68
VARIABILITY OF RESPONDING ON A CONCURRENT SCHEDULE AS A FUNCTION OF BODY WEIGHT1
McSweeney, F. K. (1974) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1974.21-357 · Cited by 67
COGNITION AND BEHAVIOR ANALYSIS
White, K. G., McCarthy, D., & Fantino, E. J. (1989) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1989.52-197 · Cited by 65
APPLYING LINEAR SYSTEMS ANALYSIS TO DYNAMIC BEHAVIOR
McDowell, J. J., Bass, R., & Kessel, R. (1992) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1992.57-377 · Cited by 64
Catania, A. C., Sagvolden, T., & Keller, K. J. (1988) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1988.49-49 · Cited by 64
Within ratio responding during fixed ratio performance
Barofsky, I., & Hurwitz, D. (1968) · Psychonomic Science · doi:10.3758/bf03327691 · Cited by 63
SHORT‐TERM MEMORY IN THE PIGEON: THE PREVIOUSLY REINFORCED RESPONSE1
Shimp, C. P. (1976) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1976.26-487 · Cited by 63
Correlated Hypothesizing and the Distinction between Contingency-Shaped and Rule-Governed Behavior
Hineline, P. N., & Wanchisen, B. A. (1989) · Springer eBooks · doi:10.1007/978-1-4757-0447-1_7 · Cited by 61
CONTINUOUS VERSUS DISCRETE DIMENSIONS OF REINFORCEMENT SCHEDULES: AN INTEGRATIVE ANALYSIS
Williams, D. C., & Johnston, J. M. (1992) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1992.58-205 · Cited by 60
Wearden, J. H., & CLARK, R. (1988) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.14.2.200 · Cited by 60
The role of information about the response-outcome relation in instrumental discrimination learning.
Rescorla, R. A. (1990) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.16.3.262 · Cited by 60
THE MAGNITUDE‐OF‐REINFORCEMENT FUNCTION IN CLOSED AND OPEN ECONOMIES
COLLIER, G. H., Johnson, D., & Morgan, C. (1992) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1992.57-81 · Cited by 60
AN ANALYTIC COMPARISON OF HERRNSTEIN'S EQUATIONS AND A MULTIVARIATE RATE EQUATION
McDowell, J. J. (1980) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1980.33-397 · Cited by 59
Parametric manipulation of interresponse-time contingency independent of reinforcement rate.
Galbicka, G., & Platt, J. R. (1986) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.12.4.371 · Cited by 59
Animal Behavior: A Continuing Synthesis
Timberlake, W. D. (1993) · Annual Review of Psychology · doi:10.1146/annurev.ps.44.020193.003331 · Cited by 59
Contiguity and contingency in instrumental conditioning
Thomas, G. V. (1983) · Learning and Motivation · doi:10.1016/0023-9690(83)90030-9 · Cited by 57
Parental favoritism toward daughters
Cronk, L. (1993) · American Scientist · Cited by 54
Galbicka, G., Kautz, M. A., & Jagers, T. (1993) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1993.60-171 · Cited by 53
DIFFERENTIATING THE BEHAVIOR OF ORGANISMS
Galbicka, G. (1988) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1988.50-343 · Cited by 52
PROBABILITY LEARNING AS A FUNCTION OF MOMENTARY REINFORCEMENT PROBABILITY1
Williams, B. A. (1972) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1972.17-363 · Cited by 50
Shimp, C. P. (1989) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1989.51-163 · Cited by 49
Information effects on the response-reinforcer association
Williams, B. A. (1978) · Animal Learning & Behavior · doi:10.3758/bf03209630 · Cited by 49
RESPONSE REQUIREMENTS AS CONSTRAINTS ON OUTPUT
Zeiler, M. D., & Buchman, I. B. (1979) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1979.32-29 · Cited by 49
Problems of modern behavior theory
SCHOENFELD, W. N. (1972) · Pavlovian Journal of Biological Science · doi:10.1007/bf03000463 · Cited by 48
Holth, P. (2012) · The Behavior Analyst · doi:10.1007/bf03392283 · Cited by 48
Shimp, C. P., Childers, L. J., & Hightower, F. A. (1990) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.16.2.200 · Cited by 48
AN ADJUSTING AVOIDANCE SCHEDULE1
Sidman, M. (1962) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1962.5-271 · Cited by 46
Effects of prior response-contingent reinforcement on superstitious behavior
Eldridge, G. D., Pear, J. J., Torgrud, L. J., et al. (1988) · Animal Learning & Behavior · doi:10.3758/bf03209077 · Cited by 44
IMPLICATIONS OF THE MATCHING LAW FOR RATIO RESPONDING
Pear, J. J. (1975) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1975.23-139 · Cited by 44
Morley, M. J., Shah, K., Bradshaw, C. M., et al. (1988) · Psychopharmacology · doi:10.1007/bf02180039 · Cited by 39
Contiguity and contingency in the acquisition and maintenance of an operant
Davis, E. R., & Platt, J. R. (1983) · Learning and Motivation · doi:10.1016/0023-9690(83)90029-2 · Cited by 38
Effect of local context of responding on human judgment of causality
Reed, P. (1992) · Memory & Cognition · doi:10.3758/bf03199589 · Cited by 34
Low doses of cis-flupentixol attenuate motor performance
Heyman, G. M., Monaghan, M. M., & Clody, D. E. (1987) · Psychopharmacology · doi:10.1007/bf00207238 · Cited by 33
Platt, J. R., & Day, R. B. (1979) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.5.4.307 · Cited by 33
Bolles, R. C. (1983) · The Psychological Record · doi:10.1007/bf03394820 · Cited by 33
BEHAVIORAL STEREOTYPY AND THE GENERALIZED MATCHING EQUATION
Pear, J. J. (1988) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1988.50-87 · Cited by 32
A physiological basis for hippocampal involvement in coding temporally discontiguous events
Deadwyler, S. A. (1985) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00001333 · Cited by 32
Wearden, J. H. (1989) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1989.51-277 · Cited by 32
Shah, K., Bradshaw, C. M., & Szabadi, E. (1991) · PubMed · Cited by 31
Working Harder for Less: Effect of Incentive Value on Force of Instrumental Response in Humans
Svartdal, F. (1993) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640749308401065 · Cited by 30
Response selection in operant learning
Staddon, J. E. R., & Zhang, Y. (1989) · Behavioural Processes · doi:10.1016/0376-6357(89)90022-3 · Cited by 30
On the hippocampus, time, and interference
Jarrard, L. E. (1985) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00001370 · Cited by 29
Memory, Temporal Discrimination, and Learned Structure in Behavior
Shimp, C. P. (1978) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/s0079-7421(08)60007-9 · Cited by 13
Cited by
sagvolden, T., johansen, E. B., aase, H., & russell, V. A. (2005) · Behavioral and Brain Sciences · doi:10.1017/s0140525x05000075 · Cited by 1,035
DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS
Lau, B., & Glimcher, P. W. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.110-04 · Cited by 485
Learning the temporal dynamics of behavior.
Machado, A. (1997) · Psychological Review · doi:10.1037//0033-295x.104.2.241 · Cited by 381
Neuroscience and approach/avoidance personality traits: A two stage (valuation–motivation) approach
Corr, P. J., & McNaughton, N. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.09.013 · Cited by 308
LINEAR-NONLINEAR-POISSON MODELS OF PRIMATE CHOICE DYNAMICS
Corrado, G. S., Sugrue, L. P., Seung, H. S., & Newsome, W. T. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.23-05 · Cited by 206
Metadata
- Authors
- Killeen, P. R.
- Venue
- Springer eBooks
- Publisher
- Springer Nature
- Published
- 1994-03-01
- Volume
- 17
- Issue
- 1
- Pages
- 105-135
- OpenAlex
- W2121904373
- Cited by
- 264
- References
- 254
- FWCI
- 9.91
- Type
- book chapter
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.