Perception, Action, and Utility: The Tangled Skein
The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Normative theories of learning and decision-making are motivated by a computational-level analysis of the task facing an animal: what should the animal do to maximize future reward? However, much of the recent excitement in this field originates in how the animal arrives at its decisions and reward predictions—-algorithmic questions about which the computational-level analysis is silent.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 2×
“The most detailed articulation of common mechanisms for inference and decision, however, has come from the work of Karl Friston and his colleagues (see Friston, 2010, for a recent review).”
BackgroundAn Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
Hierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588 · cited 1×
“The generic role commonly imputed to the perceptual system is the computation of this posterior belief (Lee and Mumford, 2003; Knill and Pouget, 2004; Friston, 2005).”
BackgroundDaw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 4×
“, 2008); of Bellman’s equation in reinforcement learning (Sutton and Barto, 1998; Dayan and Daw, 2008); of subjective expected utility theory in economics (Von Neumann and Morgenstern, 1947; Savage, 1954); and of foraging theory in behavioral ecology (McNamara and Houston, 1980; Stephens and Krebs, 1986).”
BackgroundInfluentialSelf-citationNeural Basis of a Perceptual Decision in the Parietal Cortex (Area LIP) of the Rhesus Monkey
Shadlen, M. N., & Newsome, W. T. (2001) · Journal of Neurophysiology · doi:10.1152/jn.2001.86.4.1916 · Cited by 1,755 · cited 1×
“There, Shadlen and Newsome (2001) observed direction-selective neurons that appeared to integrate motion evidence over time in a manner that predicted the timing of behavioral responses.”
BackgroundNeural correlates of decision variables in parietal cortex
Platt, M. L., & Glimcher, P. W. (1999) · Nature · doi:10.1038/22268 · Cited by 1,721
Hierarchical Bayesian inference in the visual cortex
Lee, T. S., & Mumford, D. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001434 · Cited by 1,473 · cited 2×
“The generic role commonly imputed to the perceptual system is the computation of this posterior belief (Lee and Mumford, 2003; Knill and Pouget, 2004; Friston, 2005).”
BackgroundNeuronal correlates of a perceptual decision
Newsome, W. T., Britten, K. H., & Movshon, J. A. (1989) · Nature · doi:10.1038/341052a0 · Cited by 1,177
Decision making and the avoidance of cognitive demand
Kool, W., McGuire, J. T., Rosen, Z. B., & Botvinick, M. M. (2010) · Journal of Experimental Psychology: General · doi:10.1037/a0020198 · Cited by 1,141
Matching Behavior and the Representation of Value in the Parietal Cortex
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004) · Science · doi:10.1126/science.1094765 · Cited by 1,030
Cortical microstimulation influences perceptual judgements of motion direction
Salzman, C. D., Britten, K. H., & Newsome, W. T. (1990) · Nature · doi:10.1038/346174a0 · Cited by 881
Action and behavior: a free-energy formulation
Friston, K. J., Daunizeau, J., Kilner, J., & Kiebel, S. J. (2010) · Biological Cybernetics · doi:10.1007/s00422-010-0364-z · Cited by 866
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
Self-citationReinforcement learning or active inference?
Friston, K. J., Daunizeau, J., & Kiebel, S. J. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006421 · Cited by 443
Rational approximations to rational models: alternative algorithms for category learning
Sanborn, A. N., Griffiths, T. L., & Navarro, D. J. (2010) · Psychological Review · doi:10.1037/a0020511 · Cited by 369
Speed/accuracy trade-off between the habitual and the goal-directed processes
Keramati, M., Dezfouli, A., & Piray, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002055 · Cited by 346 · cited 2×
“As suggested by Daw et al. (2005) and Keramati et al. (2011), which circumstances elicit which approach can be explained by a sort of meta-optimization over the costs (e.g. extra computation) of maintaining the full representation relative to its benefits (better statistical accuracy).”
BackgroundMethodLearning latent structure: carving nature at its joints
Gershman, S. J., & Niv, Y. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.02.008 · Cited by 326 · cited 2×
“Indeed, numerous authors (Daw et al., 2006; Dayan and Daw, 2008; Braun et al., 2010; Gershman and Niv, 2010; Rao, 2010) have argued that the full problem of decision making under perceptual uncertainty—when both the perceptual state and the utilities are unknown and non-trivial—can essentially be treated by the compos…”
BackgroundSelf-citationDecision making, movement planning and statistical decision theory
Trommershäuser, J., Maloney, L. T., & Landy, M. S. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.04.010 · Cited by 314
Braun, D. A., Mehring, C., & Wolpert, D. M. (2010) · Behavioural Brain Research · doi:10.1016/j.bbr.2009.08.031 · Cited by 210 · cited 2×
“Indeed, numerous authors (Daw et al., 2006; Dayan and Daw, 2008; Braun et al., 2010; Gershman and Niv, 2010; Rao, 2010) have argued that the full problem of decision making under perceptual uncertainty—when both the perceptual state and the utilities are unknown and non-trivial—can essentially be treated by the compos…”
BackgroundRepresentation and timing in theories of the dopamine system
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173 · cited 2×
“The brain appears to perform an analogous integration of instantaneous motion energy into an aggregate state estimate one synapse downstream from MT, in the lateral intraparietal area (LIP).”
BackgroundSelf-citation
Cited by
Whatever next? Predictive brains, situated agents, and the future of cognitive science
Clark, A. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12000477 · Cited by 5,924
Active Inference, homeostatic regulation and adaptive behavioural control
Pezzulo, G., Rigoli, F., & Friston, K. (2015) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2015.09.001 · Cited by 650
“…been proposed that use belief precisions or related constructs to modulate the balance of model based and model free controllers (Daw et al., 2005; Gershman and Daw, 2012; Lee et al., 2014; Pezzulo et al., 2013), there are some unique features of the Active Inference scheme that make novel and…”
Pezzulo, G., Rigoli, F., & Chersi, F. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00092 · Cited by 163
“Furthermore, this theory does not consider that model-based computations might have costs , linked to the cognitive effort due to planning (Gershman and Daw, 2011) and to the temporal discounting of the time required for planning (Shadmehr, 2010).”
InfluentialClark, A. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00270 · Cited by 108
First principles in the life sciences: the free-energy principle, organicism, and mechanism
Colombo, M., & Wright, C. (2021) · Synthese · doi:10.1007/s11229-018-01932-w · Cited by 101
Metadata
- Authors
- Gershman, S. J., & Daw, N. D.
- Venue
- Principles of Brain Dynamics
- Publisher
- The MIT Press
- Published
- 2012-07-06
- Pages
- 301-320
- OpenAlex
- W2626452129
- Cited by
- 36
- References
- 22
- FWCI
- 7.48
- Type
- book chapter
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.