Semi-rational models of conditioning: The case of trial order
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
This chapter considers the question of how learning adapts to changing environments, with particular reference to animal studies of operant and classical conditioning. It discusses a variety of probabilistic models, with different assumptions concerning the environment; and contrasts this type of model with a model by Kruschke (2006) which carries out local, approximate, Bayesian inference. It further suggests that it may be too early to incorporate mechanistic limitations into models of conditioning — enriching the understanding of the environment, and working with a ‘pure’ Bayesian rational analysis for that environment, may provide an alternative, and perhaps theoretically more elegant, way forward.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733 · cited 3×
“Despite this failure, the competitive model does exhibit forward blocking, albeit through a responsibilitysharing mechanism (Mackintosh, 1975) rather than a weight-sharing mechanism like Rescorla-Wagner (simulations not illustrated).”
BackgroundDaw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 3×
“…idea of normatively trading off approximate value-inference approaches characteristic of the systems has been formalized in terms of their respesctive uncertainties, and explains a wealth of data about what circumstances favor the dominance of goal-directed or habitual processes (Daw et al., 2005).”
BackgroundInfluentialSelf-citationA View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187 · cited 2×
“to reduce (or at least not increase) the error at each step (Neal and Hinton, 1998).”
BackgroundUncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · cited 1×
“One way to formalize this possibility (Yu and Dayan, 2003, 2005) is to assume that in addition to smooth Gaussian diffusion, the weights are occasionally subject to a larger shock (eg another Gaussian with width σ j σ d ).”
BackgroundSelf-citationKeeping the neural networks simple by minimizing the description length of the weights
Hinton, G. E., & van Camp, D. (1993) · Proceedings of the sixth annual conference on Computational learning theory - COLT '93 · doi:10.1145/168304.168306 · Cited by 994
Parallel incentive processing: an integrated view of amygdala function
Balleine, B. W., & Killcross, S. (2006) · Trends in Neurosciences · doi:10.1016/j.tins.2006.03.002 · Cited by 622 · cited 1×
“…(which is more cognitive in this respect and echoes our full Kalman filter's representation of interstimulus covariance) and a simpler stimulus-reward one (perhaps related to our diagonalized Kalman filter); such processes also appear to be neurally distinguishable (Balleine and Killcross, 2006).”
BackgroundLearning and selective attention
Dayan, P., Kakade, S., & Montague, P. R. (2000) · Nature Neuroscience · doi:10.1038/81504 · Cited by 495 · cited 1×
“We start by describing the Kalman filter model of conditioning (Dayan et al., 2000), which arises as the exact inference process associated with an analytically tractable, but highly simplified, Bayesian model of change.”
MethodSelf-citation
Cited by
Jones, M., & Love, B. C. (2011) · Behavioral and Brain Sciences · doi:10.1017/s0140525x10003134 · Cited by 533
Soto, F. A., Gershman, S. J., & Niv, Y. (2014) · Psychological Review · doi:10.1037/a0037018 · Cited by 81
Types of approximation for probabilistic cognition: Sampling and variational.
Sanborn, A. N. (2017) · Brain and Cognition · doi:10.1016/j.bandc.2015.06.008 · Cited by 45
“For example, a model that interpolated between a mean-field variational approximation and the full probabilistic model was used by Daw, Courville, and Dayan (2008) to produce these three trial order effects.”
Meta-learned models of cognition
Binz, M., Dasgupta, I., Jagadish, A. K., Botvinick, M., Wang, J. X., & Schulz, E. (2024) · Behavioral and Brain Sciences · doi:10.1017/s0140525x23003266 · Cited by 38
Initial Uncertainty Impacts Statistical Learning in Sound Sequence Processing
Todd, J., Provost, A., Whitson, L., & Mullens, D. (2018) · Neuroscience · doi:10.1016/j.neuroscience.2018.05.011 · Cited by 10
Metadata
- Authors
- Daw, N. D., Courville, A. C., & Dayan, P.
- Venue
- The Probabilistic Mind:
- Publisher
- Oxford University PressOxford
- Published
- 2008-03-27
- Pages
- 431-452
- OpenAlex
- W1544203215
- Cited by
- 44
- References
- 8
- FWCI
- 5.31
- Type
- book chapter
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Semi-rational models of conditioning: The case of trial order. https://app.cochat.ai/papers/10.1093/acprof:oso/9780199216093.003.0019
- The paper
- Daw, N. D., Courville, A. C., & Dayan, P. (2008). Semi-rational models of conditioning: The case of trial order. The Probabilistic Mind:. https://doi.org/10.1093/acprof%3Aoso/9780199216093.003.0019
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.