Evidence for Hyperbolic Temporal Discounting of Reward in Control of Movements
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Suppose that the purpose of a movement is to place the body in a more rewarding state. In this framework, slower movements may increase accuracy and therefore improve the probability of acquiring reward, but the longer durations of slow movements produce devaluation of reward. Here we hypothesize that the brain decides the vigor of a movement (duration and velocity) based on the expected discounted reward associated with that movement. We begin by showing that durations of saccades of varying amplitude can be accurately predicted by a model in which motor commands maximize expected discounted reward. This result suggests that reward is temporally discounted even in timescales of tens of milliseconds. One interpretation of temporal discounting is that the true objective of the brain is to maximize the rate of reward—which is equivalent to a specific form of hyperbolic discounting. A consequence of this idea is that the vigor of saccades should change as one alters the intertrial intervals between movements. We find experimentally that in healthy humans, as intertrial intervals are varied, saccade peak velocities and durations change on a trial-by-trial basis precisely as predicted by a model in which the objective is to maximize the rate of reward. Our results are inconsistent with theories in which reward is discounted exponentially. We suggest that there exists a single cost, rate of reward, which provides a unifying principle that may govern control of movements in timescales of milliseconds, as well as decision making in timescales of seconds to years.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The information capacity of the human motor system in controlling the amplitude of movement.
Fitts, P. M. (1954) · Journal of Experimental Psychology · doi:10.1037/h0055392 · Cited by 6,466 · cited 1×
“Fast movements, however, are more variable (Fitts, 1954; Schmidt et al., 1979), reducing the probability of success for the movement.”
BackgroundSignal-dependent noise determines motor planning
Harris, C. M., & Wolpert, D. M. (1998) · Nature · doi:10.1038/29528 · Cited by 2,498 · cited 1×
“An early model of saccades (Harris and Wolpert, 1998) showed that saccade trajectories for a given duration are well predicted by a model in which motor commands are selected to minimize endpoint variance in the presence of signal-dependent noise (term t in Eq.”
BackgroundToward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450 · cited 2×
“(3) For example, exponential temporal discounting is routinely used in a form of reinforcement learning known as temporal difference learning (Sutton and Barto, 1981), which provides a prom- inent theory of learning in the basal ganglia (Schultz et al., 1997).”
BackgroundTonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · cited 3×
“An alternative interpretation is that temporal discounting reflects a desire to maximize the rate of reward acquisition, rather than the absolute value of each acquired reward (Kacelnik, 1997; Daw, 2003; Niv et al., 2007).”
BackgroundResponses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Why Don't We Move Faster? Parkinson's Disease, Movement Vigor, and Implicit Motivation
Mazzoni, P., Hristova, A., & Krakauer, J. W. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0264-07.2007 · Cited by 557
Cited by
Mesolimbic dopamine signals the value of work
Hamid, A. A., Pettibone, J. R., Mabrouk, O. S., Hetrick, V. L., Schmidt, R., Vander Weele, C. M., Kennedy, R. T., Aragona, B. J., & Berke, J. D. (2016) · Nature Neuroscience · doi:10.1038/nn.4173 · Cited by 824
The Role of Variability in Motor Learning
Dhawale, A. K., Smith, M. A., & Ölveczky, B. P. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031548 · Cited by 516
Reward Pays the Cost of Noise Reduction in Motor and Cognitive Control
Manohar, S. G., Chong, T. T.-J., Apps, M. A. J., Batla, A., Stamelou, M., Jarman, P. R., Bhatia, K. P., & Husain, M. (2015) · Current Biology · doi:10.1016/j.cub.2015.05.038 · Cited by 317
The basal ganglia: from motor commands to the control of vigor
Dudman, J. T., & Krakauer, J. W. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.02.005 · Cited by 281
A Representation of Effort in Decision-Making and Motor Control
Shadmehr, R., Huang, H. J., & Ahmed, A. A. (2016) · Current Biology · doi:10.1016/j.cub.2016.05.065 · Cited by 267
Self-citation
Metadata
- Authors
- Haith, A. M., Reppert, T. R., & Shadmehr, R.
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2012-08-22
- Volume
- 32
- Issue
- 34
- Pages
- 11727-11736
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- OpenAlex
- W1983408848
- Cited by
- 132
- References
- 6
- FWCI
- 3.84
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evidence for Hyperbolic Temporal Discounting of Reward in Control of Movements. https://app.cochat.ai/papers/10.1523/jneurosci.0424-12.2012
- The paper
- Haith, A. M., Reppert, T. R., & Shadmehr, R. (2012). Evidence for Hyperbolic Temporal Discounting of Reward in Control of Movements. The Journal of Neuroscience. https://doi.org/10.1523/jneurosci.0424-12.2012
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.