Humans Can Adopt Optimal Discounting Strategy under Real-Time Constraints
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Critical to our many daily choices between larger delayed rewards, and smaller more immediate rewards, are the shape and the steepness of the function that discounts rewards with time. Although research in artificial intelligence favors exponential discounting in uncertain environments, studies with humans and animals have consistently shown hyperbolic discounting. We investigated how humans perform in a reward decision task with temporal constraints, in which each choice affects the time remaining for later trials, and in which the delays vary at each trial. We demonstrated that most of our subjects adopted exponential discounting in this experiment. Further, we confirmed analytically that exponential discounting, with a decay rate comparable to that used by our subjects, maximized the total reward gain in our task. Our results suggest that the particular shape and steepness of temporal discounting is determined by the task that the subject is facing, and question the notion of hyperbolic reward discounting as a universal principle.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Golden eggs and hyperbolic discounting
Laibson, D. (1997) · The Quarterly Journal of Economics · doi:10.1162/003355397555253 · Cited by 6,122
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089 · cited 1×
“If reward prediction at a larger time scale is required, as in questionnaire-based human experiments, the frontal cortex would be additionally recruited [39].”
BackgroundSpecious reward: A behavioral theory of impulsiveness and impulse control.
Ainslie, G. (1975) · Psychological Bulletin · doi:10.1037/h0076860 · Cited by 2,708 · cited 1×
“Two main classes of models that characterize the shape of reward discounting have been proposed: exponential [2–4] and hyperbolic [5–13].”
BackgroundOut of Control: Visceral Influences on Behavior
Loewenstein, G. (1996) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1996.0028 · Cited by 2,610
An economic theory of self-control.
Thaler, R. H., & Shefrin, H. M. (1981) · Journal of Political Economy · doi:10.1086/260971 · Cited by 2,203
Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls.
Kirby, K. N., Petry, N. M., & Bickel, W. K. (1999) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.128.1.78 · Cited by 2,171
Impulsivity and cigarette smoking: delay discounting in current, never, and ex-smokers
Bickel, W. K., Odum, A. L., & Madden, G. J. (1999) · Psychopharmacology · doi:10.1007/pl00005490 · Cited by 1,433
SUBJECTIVE PROBABILITY AND DELAY
Rachlin, H., Raineri, A., & Cross, D. (1991) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1991.55-233 · Cited by 1,341
Madden, G. J., Petry, N. M., Badger, G. J., & Bickel, W. K. (1997) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.5.3.256 · Cited by 911
Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops
Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S. (2004) · Nature Neuroscience · doi:10.1038/nn1279 · Cited by 779 · cited 1×
“What may be the possible neural correlates of exponential or hyperbolic discounting? We have previously found that parallel cortico–basal ganglia loops are involved in reward prediction with different discounting factors [38].”
BackgroundSelf-citationImpulsivity and rapid discounting of delayed hypothetical rewards in cocaine-dependent individuals.
Coffey, S. F., Gudleski, G. D., Saladin, M. E., & Brady, K. T. (2003) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.11.1.18 · Cited by 625 · cited 1×
“Individuals with impulse-control disorders, as well as heroin, alcohol, cigarette-, and cocaine-addicted individuals, have steeper discounting functions than controls [10,26– 29].”
BackgroundHyperbolic temporal discounting in social drinkers and problem drinkers.
Vuchinich, R. E., & Simpson, C. A. (1998) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.6.3.292 · Cited by 624
Meta-learning in Reinforcement Learning
Schweighofer, N., & Doya, K. (2003) · Neural Networks · doi:10.1016/s0893-6080(02)00228-9 · Cited by 261 · cited 2×
“Thus, the discount rate should be carefully adjusted to maximize total gain in task situations of repeated forced choices with varying delays to the rewards and limited total time [14,15].”
BackgroundSelf-citation
Cited by
Reinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 704
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642
“In deciding between actions with less-than-average rewards, an action with smaller reward in shorter time can be more appropriate because it allows moving on to the next action earlier, resulting in an apparently impulsive choic”
BackgroundSelf-citationSeparate Valuation Subsystems for Delay and Effort Decision Costs
Prévost, C., Pessiglione, M., Météreau, E., Cléry-Melin, M.-L., & Dreher, J.-C. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.2752-10.2010 · Cited by 499
Rosati, A. G., Stevens, J. R., Hare, B., & Hauser, M. D. (2007) · Current Biology · doi:10.1016/j.cub.2007.08.033 · Cited by 356
Serotonin, inhibition, and negative mood
Dayan, P., & Huys, Q. J. M. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0040004 · Cited by 263
Metadata
- Authors
- Schweighofer, N., Shishida, K., Han, C. E., Okamoto, Y., Tanaka, S. C., Yamawaki, S., & Doya, K.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2006
- Volume
- 2
- Issue
- 11
- Pages
- e152
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2112241171
- Cited by
- 82
- References
- 13
- FWCI
- 4.52
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.