A theoretical account of cognitive effects in delay discounting
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
Although delay discounting, the attenuation of the value of future rewards, is a robust finding, the mechanism of discounting is not known. We propose a potential mechanism for delay discounting such that discounting emerges from a search process that is trying to determine what rewards will be available in the future. In this theory, the delay dependence of the discounting of future expected rewards arises from three assumptions. First, that the evaluation of outcomes involves a search process. Second, that the value is assigned to an outcome proportionally to how easy it is to find. Third, that outcomes that are less delayed are typically easier for the search process to find. By relaxing this third assumption (e.g. by assuming that episodically‐cued outcomes are easier to find), our model suggests that it is possible to dissociate discounting from delay. Our theory thereby explains the empirical result that discounting is slower to episodically‐imagined outcomes, because these outcomes are easier for the search process to find. Additionally, the theory explains why improving cognitive resources such as working memory slows discounting, by improving searches and thereby making rewards easier to find. The three assumptions outlined here are likely to be instantiated during deliberative decision‐making, but are unlikely in habitual decision‐making. We model two simple implementations of this theory and show that they unify empirical results about the role of cognitive function in delay discounting, and make new neural, behavioral, and pharmacological predictions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270 · cited 2×
“Model 2: Hopfield network This simulation used a standard Hopfield network (Hopfield, 1982; Hertz et al., 1991).”
MethodReinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Golden eggs and hyperbolic discounting
Laibson, D. (1997) · The Quarterly Journal of Economics · doi:10.1162/003355397555253 · Cited by 6,122
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089 · cited 2×
“, 2006), exponential with a bonus for immediate rewards (Laibson, 1997; McClure et al., 2004), and the sum of multiple exponentials (Sozou, 1998; Kurth-Nelson &”
BackgroundThe Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks
Ratcliff, R., & McKoon, G. (2008) · Neural Computation · doi:10.1162/neco.2008.12-06-420 · Cited by 3,059 · cited 2×
“Forcing decisions to terminate early produces more inconsistent results (Gold & Shadlen, 2002; Ratcliff & McKoon, 2008), but can also lead to falling back on inherent biases and changing discounting functions (Simonson & Tversky, 1992; Lauwereyns, 2010).”
BackgroundTrope, Y., & Liberman, N. (2003) · Psychological Review · doi:10.1037/0033-295x.110.3.403 · Cited by 3,006
Buckner, R. L., & Carroll, D. C. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.11.004 · Cited by 2,691
The Medial Temporal Lobe and Recognition Memory
Eichenbaum, H., Yonelinas, A. P., & Ranganath, C. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.30.051606.094328 · Cited by 2,580
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 6×
“…from multiple decision-making processes, which use different information-processing algorithms, including Pavlovian, deliberative (model-based) and habit (model-free) (Daw et al., 2005; Huys, 2007; Redish et al., 2008; van der Meer & Redish, 2010; van der Meer et al., 2012; Montague et al., 2012).”
BackgroundMethodInfluentialRemembering the past to imagine the future: the prospective brain
Schacter, D. L., Addis, D. R., & Buckner, R. L. (2007) · Nature Reviews Neuroscience · doi:10.1038/nrn2213 · Cited by 2,394 · cited 8×
“…decision-making in particular has been proposed to depend on search processes that proceed through potential futures (Johnson & Redish, 2007; van der Meer et al., 2012) and ⁄ or the creation of imagined expectations (episodic future thinking) (Atance & O’Neill, 2001; Schacter et al., 2007).”
BackgroundInfluentialMemory deficits associated with senescence: A neurophysiological and behavioral study in the rat.
Barnes, C. A. (1979) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0077579 · Cited by 2,088
Addis, D. R., Wong, A. T., & Schacter, D. L. (2007) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.10.016 · Cited by 1,949 · cited 4×
“the evidence that subjects with hippocampal or prefrontal damage cannot accomplish episodic future thinking (Addis et al., 2007; Hassabis et al., 2007) implies that any discounting functions that they”
BackgroundInfluentialReconstruction of automobile destruction: An example of the interaction between language and memory
Loftus, E. F., & Palmer, J. C. (1974) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(74)80011-3 · Cited by 1,909
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598 · cited 1×
“…brain network that has learned to associate which kinds of situations are valuable and which are not, probably involving the orbitofrontal cortex (Padoa-Schioppa & Assad, 2006; Zald & Rauch, 2006; Schoenbaum et al., 2009) and ventral striatum (Pennartz et al., 2009; Roesch et al., 2009;…”
BackgroundThe principal features and mechanisms of dopamine modulation in the prefrontal cortex
Seamans, J. K., & Yang, C. R. (2004) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2004.05.006 · Cited by 1,592 · cited 1×
“Several models (Seamans & Yang, 2004; Yamashita & Tanaka, 2005; Redish et al., 2007; Durstewitz et al., 2010) have suggested that the basin depth in prefrontal and hippocampal representations depends on dopaminergic tone, with low tonic levels of dopamine implying shallow basins and fragile…”
BackgroundPatients with hippocampal amnesia cannot imagine new experiences
Hassabis, D., Kumaran, D., Vann, S. D., & Maguire, E. A. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0610561104 · Cited by 1,414
Visual fixations and the computation and comparison of value in simple choice
Krajbich, I., Armel, C., & Rangel, A. (2010) · Nature Neuroscience · doi:10.1038/nn.2635 · Cited by 1,408
Self-control relies on glucose as a limited energy source: Willpower is more than a metaphor.
Gailliot, M. T., Baumeister, R. F., DeWall, C. N., Maner, J. K., Plant, E. A., Tice, D. M., Brewer, L. E., & Schmeichel, B. J. (2007) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.92.2.325 · Cited by 1,297 · cited 4×
“, 2009), compromising cognitive abilities, such as providing a cognitive load (Vohs & Faber, 2007) or depleting cognitive resources (Baumeister et al., 1994; Hinson et al., 2003; Gailliot et al., 2007), and training cognitive abilities (Bickel et al.”
BackgroundInfluentialProspection: experiencing the future
Gilbert, D. T., & Wilson, T. D. (2007) · Science · doi:10.1126/science.1144161 · Cited by 1,238 · cited 2×
“By relaxing this third assumption (e.g. by assuming that episodically-cued outcomes are easier to find), our model suggests that it is possible to dissociate discounting from delay.”
BackgroundResultToward a behavioral economic understanding of drug dependence: delay discounting processes
Bickel, W. K., & Marsch, L. A. (2001) · Addiction · doi:10.1046/j.1360-0443.2001.961736.x · Cited by 1,117
Self-citationNeural ensembles in CA3 transiently encode paths forward of the animal at a decision point
Johnson, A., & Redish, A. D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3761-07.2007 · Cited by 1,003 · cited 13×
“In both human and non-human animals, the ability to wait for a larger reward is positively related to self-control abilities (Mischel & Underwood, 1974; Baumeister et al., 1994; Peters & Büchel, 2011) and inversely related to addiction liabilities (Ainslie, 2001; Bickel & Marsch, 2001; Odum et al., 2002; Perry et al.…”
BackgroundMethodResultInfluentialSelf-citationZhang, K. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-06-02112.1996 · Cited by 1,001 · cited 2×
“, 2007; Buckner, 2010), brain structures that are often modeled as autoassociators with attractor dynamics (Wilson & Cowan, 1972; Zhang, 1996; Redish, 1999; Durstewitz et al., 2000).”
BackgroundPeters, J., & Büchel, C. (2010) · Neuron · doi:10.1016/j.neuron.2010.03.026 · Cited by 871 · cited 9×
“…rates depend on the episodic nature of the potential options in a questionnaire – if the delayed option is primed with an episodic cue, then subjects are more likely to select it, leading to a decrease in the rate of decay of the discounting function (Peters & Büchel, 2010; Benoit et al., 2011).”
BackgroundInfluentialEpisodic Simulation of Future Events: Concepts, Data, and Applications
Schacter, D. L., Addis, D. R., & Buckner, R. L. (2008) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1440.001 · Cited by 840
Remember the Future: Working Memory Training Decreases Delay Discounting Among Stimulant Addicts
Bickel, W. K., Yi, R., Landes, R. D., Hill, P. F., & Baxter, C. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2010.08.017 · Cited by 704 · cited 8×
“…et al., 2006; Burks et al., 2009), compromising cognitive abilities, such as providing a cognitive load (Vohs & Faber, 2007) or depleting cognitive resources (Baumeister et al., 1994; Hinson et al., 2003; Gailliot et al., 2007), and training cognitive abilities (Bickel et al., 2011).”
BackgroundInfluentialSelf-citationThe neural mechanisms of inter-temporal decision-making: understanding variability
Peters, J., & Büchel, C. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.03.002 · Cited by 693 · cited 1×
“…the ability to wait for a larger reward is positively related to self-control abilities (Mischel & Underwood, 1974; Baumeister et al., 1994; Peters & Büchel, 2011) and inversely related to addiction liabilities (Ainslie, 2001; Bickel & Marsch, 2001; Odum et al., 2002; Perry et al., 2005;…”
BackgroundImpulsive decision making and working memory.
Hinson, J. M., Jameson, T. L., & Whitney, P. (2003) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.29.2.298 · Cited by 595
A new perspective on the role of the orbitofrontal cortex in adaptive behaviour
Schoenbaum, G., Roesch, M. R., Stalnaker, T. A., & Takahashi, Y. K. (2009) · Nature reviews. Neuroscience · doi:10.1038/nrn2753 · Cited by 585
A unified framework for addiction: Vulnerabilities in the decision process
Redish, A. D., Jensen, S., & Johnson, A. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0800472x · Cited by 501 · cited 3×
“Current theories of decision-making systems suggest that there are multiple systems that can drive decision-making (Daw et al., 2005; Redish et al., 2008; van der Meer et al., 2012; Montague et al., 2012), including both deliberative and non-deliberative (habit) systems.”
BackgroundMethodSelf-citationHuys, Q. J. M., Eshel, N., O'Nions, E., Sheridan, L., Dayan, P., & Roiser, J. P. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002410 · Cited by 439
A neural mechanism mediating the impact of episodic prospection on farsighted decisions
Benoit, R. G., Gilbert, S. J., & Burgess, P. W. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6559-10.2011 · Cited by 422
The Role of the Hippocampus in Prediction and Imagination
Buckner, R. L. (2010) · Annual Review of Psychology · doi:10.1146/annurev.psych.60.110707.163508 · Cited by 419 · cited 2×
“…future thinking is known to depend on the hippocampus and prefrontal cortex (Addis et al., 2007; Hassabis et al., 2007; Schacter et al., 2007; Buckner, 2010), brain structures that are often modeled as autoassociators with attractor dynamics (Wilson & Cowan, 1972; Zhang, 1996; Redish, 1999;…”
BackgroundPerry, J. L., Larson, E. B., German, J. P., Madden, G. J., & Carroll, M. E. (2005) · Psychopharmacology · doi:10.1007/s00213-004-1994-4 · Cited by 388 · cited 2×
“related to addiction liabilities (Ainslie, 2001; Bickel & Marsch, 2001; Odum et al., 2002; Perry et al., 2005; Anker et al., 2009; Heyman, 2009; Stanger et al., 2011).”
BackgroundRedish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.784 · Cited by 346 · cited 4×
“Several models (Seamans & Yang, 2004; Yamashita & Tanaka, 2005; Redish et al., 2007; Durstewitz et al., 2010) have suggested that the basin depth in prefrontal and hippocampal representations depends on dopaminergic tone, with low tonic levels of dopamine implying shallow basins and fragile…”
BackgroundInfluentialSelf-citationRecent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1025696116075 · Cited by 317
Howard, M. W., Fotedar, M. S., Datey, A. V., & Hasselmo, M. E. (2005) · Psychological Review · doi:10.1037/0033-295x.112.1.75 · Cited by 283 · cited 4×
“…& Carroll, 2007)], (ii) that temporally-delayed rewards are more difficult to find [because they are more contextually separated (Rachlin, 2004; Howard et al., 2005) and thus farther away in the representation space], and (iii) that one calculates expected value as a function of the ability to…”
BackgroundInfluentialA normative perspective on motivation
NIV, Y., JOEL, D., & DAYAN, P. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.06.010 · Cited by 251
Contributions of Striatal Subregions to Place and Response Learning
Yin, H. H., & Knowlton, B. J. (2004) · Learning & Memory · doi:10.1101/lm.81004 · Cited by 238 · cited 1×
“Different training regimens lead to the selection of different strategies (Restle, 1957) that depend on different brain structures (Barnes, 1979; Packard & McGaugh, 1996; Yin & Knowlton, 2004) and different computations (Daw et al., 2005; van der Meer & Redish, 2010).”
BackgroundDiscounting of delayed health gains and losses by current, never- and ex-smokers of cigarettes
Odum, A. L., Madden, G. J., & Bickel, W. K. (2002) · Nicotine & Tobacco Research · doi:10.1080/14622200210141257 · Cited by 234 · cited 2×
“related to addiction liabilities (Ainslie, 2001; Bickel & Marsch, 2001; Odum et al., 2002; Perry et al., 2005; Anker et al., 2009; Heyman, 2009; Stanger et al., 2011).”
BackgroundSelf-citationCorticostriatal Interactions during Learning, Memory Processing, and Decision Making
Pennartz, C. M. A., Berke, J. D., Graybiel, A. M., Ito, R., Lansink, C. S., van der Meer, M., Redish, A. D., Smith, K. S., & Voorn, P. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3177-09.2009 · Cited by 213
Self-citationImpulsivity predicts the escalation of cocaine self-administration in rats
Anker, J. J., Perry, J. L., Gliddon, L. A., & Carroll, M. E. (2009) · Pharmacology Biochemistry and Behavior · doi:10.1016/j.pbb.2009.05.013 · Cited by 188
Bickel, W. K., Yi, R., Kowal, B. P., & Gatchalian, K. M. (2008) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2008.03.009 · Cited by 186
Self-citationStill at the Choice‐Point: Action Selection and Initiation in Instrumental Conditioning
BALLEINE, B. W., & OSTLUND, S. B. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.006 · Cited by 184
Cited by
Exploration versus exploitation in space, mind, and society
Hills, T. T., Todd, P. M., Lazer, D., Redish, A. D., & Couzin, I. D. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.10.004 · Cited by 549
Self-citationRedish, A. D. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2015.30 · Cited by 479
Self-citationOver the river, through the woods: cognitive maps in the hippocampus and orbitofrontal cortex
Wikenheiser, A. M., & Schoenbaum, G. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2016.56 · Cited by 359
Daniel (2013) · doi:10.1177/0956797613488780 · Cited by 336
Behavioral components of impulsivity.
Stahl, C., Voss, A., Schmitz, F., Nuszbaum, M., Tüscher, O., Lieb, K., & Klauer, K. C. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033981 · Cited by 319
Metadata
- Authors
- Kurth‐Nelson, Z., Bickel, W., & Redish, A. D.
- Venue
- European Journal of Neuroscience
- Publisher
- Wiley
- Published
- 2012-04
- Volume
- 35
- Issue
- 7
- Pages
- 1052-1064
- ISSN
- 0953-816X (ISSN-L)
- 1460-9568
- OpenAlex
- W2154356623
- Cited by
- 97
- References
- 44
- FWCI
- 5.67
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.