Dopamine, Learning, and Impulsivity: ABiological Account of Attention-Deficit/Hyperactivity Disorder
The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Attention-deficit/hyperactivity disorder (ADHD) affects up to 10% of school-age children. The impulsivity which is seen as its core feature persists over years, yet experimental measures of impulsivity can be altered in a single session. In this study, we tested the theory that both the persistence and the variability of impulsivity could be the result of abnormalities in learning mechanisms and environment. METHOD: We extended an existing model of the role of dopamine in operant conditioning to address the delayed response time task, which is one of the standard tests for impulsivity in ADHD. In this task, subjects choose between immediate responding for a small reinforcer and later responding for a larger one. We studied the influence on impulsivity of four key parameters of the model: The learning rate, discount factor, brittleness, and action bias. RESULTS: The behavior of the model is broadly comparable with electrophysiological (monkey) and behavioral (ADHD and normal) data. Variations in any of the parameters can cause impulsivity. All parameters except the discount factor show inverted U-shaped curves for their effects on impulsivity, suggesting, for example, how either hyper- or hypofunctioning of dopamine can cause impulsivity. The model suggests how decision making can be affected by environmental unpredictability, and thus offers an account of one aspect of the natural history of ADHD. CONCLUSIONS: Some types of ADHD may be caused by specific deficits in reinforcement learning and in the use of learned lessons. Environmental factors can interact with these deficits to delay maturation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
A Systematic Method for Clinical Description and Classification of Personality Variants: A Proposal
Cloninger, C. R. (1987) · Archives of General Psychiatry · doi:10.1001/archpsyc.1987.01800180093014 · Cited by 3,678
Specious reward: A behavioral theory of impulsiveness and impulse control.
Ainslie, G. (1975) · Psychological Bulletin · doi:10.1037/h0076860 · Cited by 2,708 · cited 2×
“However, a different kind of curve, the hyperbolic, better describes the discounting found in many psychological studies (Ainslie 1975; but see also Kacelnik 1997; Kirby and Herrnstein 1995; Madden et al. 2003; Monterosso and Ainslie 1999).”
BackgroundHeroin addicts have higher discount rates for delayed rewards than non-drug-using controls.
Kirby, K. N., Petry, N. M., & Bickel, W. K. (1999) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.128.1.78 · Cited by 2,171
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Self-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 2×
“However , there is, as yet, no evidence that children show preference reversals in the DRTT, and so, for simplicity, we preserve the exponential discounting that Montague et al. (1996) used in their model.”
MethodSelf-citationEvenden, J. L. (1999) · Psychopharmacology · doi:10.1007/pl00005481 · Cited by 2,044 · cited 1×
“Experimental tests used to access these include delayed reward, go-no go, stop tasks, response time in the uncertain visual discrimination test, and delayed response time tasks (DRTT) (Evenden 1999; Rubia et al. 2001).”
MethodPashler, H. (1997) · doi:10.7551/mitpress/5677.001.0001 · Cited by 1,607
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Self-citationBiederman, J. (2000) · American Journal of Psychiatry · doi:10.1176/appi.ajp.157.5.816 · Cited by 1,498 · cited 1×
“ADHD symptoms decrease as a child grows older (Biederman et al. 2000; Scahill and Schwab-Stone 2000), though some of the apparent decrease may result from the use of age-inappropriate diagnostic criteria.”
BackgroundRubia, K., Overmeyer, S., Taylor, E., Brammer, M., Williams, S. C. R., Simmons, A., & Bullmore, E. T. (1999) · American Journal of Psychiatry · doi:10.1176/ajp.156.6.891 · Cited by 1,010
Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events
Horvitz, J. C. (2000) · Neuroscience · doi:10.1016/s0306-4522(00)00019-1 · Cited by 967
LEVY, F., HAY, D. A., McSTEPHEN, M., WOOD, C., & WALDMAN, I. (1997) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-199706000-00009 · Cited by 956 · cited 1×
“1999); unfortunately, we do not know the effect of birth order, which could shed light on the well-replicated absence of significant additive contributions from shared environmental factors (Levy et al. 1997).”
BackgroundTemporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · cited 1×
“It has also been used to model human (fMRI) data in reward learning (McClure et al. 2003; O’Doherty et al. 2003, 2004).”
MethodBush, G., Frazier, J. A., Rauch, S. L., Seidman, L. J., Whalen, P. J., Jenike, M. A., Rosen, B. R., & Biederman, J. (1999) · Biological Psychiatry · doi:10.1016/s0006-3223(99)00083-9 · Cited by 866
Delay discounting of real and hypothetical rewards.
Madden, G. J., Begotka, A. M., Raiff, B. R., & Kastern, L. L. (2003) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.11.2.139 · Cited by 660
Amygdala circuitry in attentional and representational processes
Holland, P. C., & Gallagher, M. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01271-6 · Cited by 618
Preference Reversals Due to Myopic Discounting of Delayed Reward
Kirby, K. N., & Herrnstein, R. J. (1995) · Psychological Science · doi:10.1111/j.1467-9280.1995.tb00311.x · Cited by 591
Barkley, R. A. (1997) · Psychological Bulletin · doi:10.1037//0033-2909.121.1.65 · Cited by 580 · cited 2×
“Many researchers have suggested that the central deficit in ADHD is impulsivity (Barkley 1997; Tannock 1998).”
BackgroundHyperactivity and Delay Aversion—I. The Effect of Delay on Choice
Sonuga‐Barke, E. J. S., Taylor, E., Sembi, S., & Smith, J. (1992) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.1992.tb00874.x · Cited by 578
McBride, W. J., Murphy, J. M., & Ikemoto, S. (1999) · Behavioural Brain Research · doi:10.1016/s0166-4328(99)00022-4 · Cited by 574
Importance of unpredictability for reward responses in primate dopamine neurons
Mirenowicz, J., & Schultz, W. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.72.2.1024 · Cited by 569
Altered reinforcement mechanisms in attention-deficit/hyperactivity disorder
Sagvolden, T., Aase, H., Zeiner, P., & Berger, D. (1998) · Behavioural Brain Research · doi:10.1016/s0166-4328(97)00170-8 · Cited by 345 · cited 1×
“Indeed, Sagvolden et al. (1998) have explicitly demonstrated this effect, which they describe as a " shorter and steeper delay gradient " in ADHD children, and they have proposed that this gradient leads to the development of overactivity, increased behavioral variability, motor impulsiveness, and…”
BackgroundHigh Midbrain [ 18 F]DOPA Accumulation in Children With Attention Deficit Hyperactivity Disorder
Ernst, M., Zametkin, A. J., Matochik, J. A., Pascualvaca, D., Jons, P. H., & Cohen, R. M. (1999) · American Journal of Psychiatry · doi:10.1176/ajp.156.8.1209 · Cited by 256 · cited 1×
“Elevated midbrain accumulation of dopamine precursors has been found in adolescents with ADHD (Ernst et al. 1999).”
BackgroundAttention Deficit Disorder With Hyperactivity: A Specific Hypothesis of Reward Dysfunction
HAENLEIN, M., & CAUL, W. F. (1987) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-198705000-00014 · Cited by 213
Doyle, A. E., Biederman, J., Seidman, L. J., Weber, W., & Faraone, S. V. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.68.3.477 · Cited by 209
Effects of methamphetamine on the adjusting amount procedure, a model of impulsive behavior in rats
Richards, J. B., Sabol, K. E., & de Wit, H. (1999) · Psychopharmacology · doi:10.1007/pl00005488 · Cited by 203
Motivational Effects on Inhibitory Control in Children With ADHD
SLUSAREK, M., VELLING, S., BUNK, D., & EGGERS, C. (2001) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-200103000-00016 · Cited by 184 · cited 2×
“Their data match the model's, if one arbitrarily assumes that their thorough explanation had an effect comparable to preliminary trials (we had no way of verbally priming our model, and preliminary trials are often included in such studies, e.g., 30 in Slusarek et al. (2001)).”
BackgroundResultCador, M., Taylor, J. R., & Robbins, T. W. (1991) · Psychopharmacology · doi:10.1007/bf02246039 · Cited by 158 · cited 1×
“The possibility that stimulants act by altering the processing of reinforcement in the brain has experimental support from many studies (Cador et al. 1991; Evenden 1999; see McBride et al. 1999; Richards et al. 1999).”
ResultReed, P., Mitchell, C., & Nokes, T. (1996) · Animal Learning & Behavior · doi:10.3758/bf03198952 · Cited by 59
Methylphenidate and cognitive perseveration in hyperactive children
Tannock, R., & Schachar, R. (1992) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.1992.tb00940.x · Cited by 59
Cited by
Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning
Tsai, H.-C., Zhang, F., Adamantidis, A., Stuber, G. D., Bonci, A., de Lecea, L., & Deisseroth, K. (2009) · Science · doi:10.1126/science.1168878 · Cited by 1,266
Keller, M. C., & Miller, G. (2006) · Behavioral and Brain Sciences · doi:10.1017/s0140525x06009095 · Cited by 540
Tripp, G., & Wickens, J. R. (2009) · Neuropharmacology · doi:10.1016/j.neuropharm.2009.07.026 · Cited by 492
“A further two models approach ADHD from a neurocomputational perspective (Frank, 2005; Frank et al., 2007b; Williams and Dayan, 2005).”
BackgroundOpponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456
Self-citationSerotonin in Affective Control
Dayan, P., & Huys, Q. J. M. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135607 · Cited by 381
Self-citation
Metadata
- Authors
- Williams, J., & Dayan, P.
- Venue
- Journal of Child and Adolescent Psychopharmacology
- Publisher
- SAGE Publications
- Published
- 2005-04
- Volume
- 15
- Issue
- 2
- Pages
- 160-179
- ISSN
- 1044-5463 (ISSN-L)
- 1557-8992
- OpenAlex
- W2068367895
- Cited by
- 78
- References
- 31
- FWCI
- 3.12
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.