CoChat
Sign Up
Cited by 15Substance Abuse Treatment and OutcomesNeural and Behavioral Psychology StudiesGambling Behavior and Treatments

Active learning impairments in substance use disorders when resolving the explore-exploit dilemma: A replication and extension of previous computational modeling results

Open access
Taylor, Samuel, Lavalley, Claire A., Hakimi, Navid, Stewart, Jennifer L., Ironside, Maria, Zheng, Haixia, White, Evan, Guinjoan, Salvador +2 more·2023·Drug and Alcohol Dependence
DOI 10.1016/j.drugalcdep.2023.110945PMC PMC10635739

Published version of a preprint

Open this paper in CoChat — free account

Abstract

Background Substance use disorders (SUDs) represent a major public health risk. Yet, our understanding of the mechanisms that maintain these disorders remains incomplete. In a recent computational modeling study, we found initial evidence that SUDs are associated with slower learning rates from negative outcomes and less value-sensitive choice (low "action precision"), which could help explain continued substance use despite harmful consequences. Methods Here we aimed to replicate and extend these results in a pre-registered study with a new sample of 168 individuals with SUDs and 99 healthy comparisons (HCs). We performed the same computational modeling and group comparisons as in our prior report (doi: 10.1016/j.drugalcdep.2020.108208) to confirm previously observed effects. After completing all pre-registered replication analyses, we then combined the previous and current datasets (N = 468) to assess whether differences were transdiagnostic or driven by specific disorders. Results Replicating prior results, SUDs showed slower learning rates for negative outcomes in both Bayesian and frequentist analyses (partial η 2=.02). Previously observed differences in action precision were not confirmed. Learning rates for positive outcomes were also similar between groups. Logistic regressions including all computational parameters as predictors in the combined datasets could differentiate several specific disorders from HCs, but could not differentiate most disorders from each other. Conclusions These results provide robust evidence that individuals with SUDs adjust behavior more slowly in the face of negative outcomes than HCs. They also suggest this effect is common across several different SUDs. Future research should examine its neural basis and whether learning rates could represent a new treatment target or moderator of treatment outcome.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.drugalcdep.2023.110945

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In an experiment, patients with addictions required more negative experience than neurotypical volunteers to abandon previously reinforced behavior under decreasing reward and reversal of reinforcement rule.

    “пациентам с зависимостями требовалось накопить больше негативного опыта, чем нейротипичным добровольцам, для того чтобы отказаться от ранее подкрепляемого способа поведения при постепенном уменьшении размера вознаграждения и инверсии правила подкрепления (Taylor et al., 2023).”

    From Когнитивные механизмы дисфункционального принятия решений при психических расстройствах (2025) · cites this paper as Taylor et al., 2023

    The Abstract and Discussion report that participants with substance use disorders learned more slowly from negative outcomes than healthy comparisons, supporting the claim that they needed more negative experience to adjust their choices. But the Methods (Section 2.2) describe a three-armed bandit task; our copy does not report gradually decreasing reward amounts or a reversal of the reinforcement rule. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 40

  1. [1]1. Darke S, Kaye S, McKetin R, Duflou J. Major physical and psychological harms of methamphetamine use. Drug Alcohol Rev. 2008;27(3):253-62.
  2. [2]2. Zweben JE, Cohen JB, Christian D, Galloway GP, Salinardi M, Parent D, et al. Psychiatric symptoms in methamphetamine users. Am J Addict. 2004;13(2):181-90.
  3. [3]3. May AC, Aupperle RL, Stewart JL. Dark Times: The Role of Negative Reinforcement in Methamphetamine Addiction. Front Psychiatry. 2020;11:114.
  4. [4]4. Darke S, Kaye S, Duflou J, Lappin J. Completed Suicide Among Methamphetamine Users: A National Study. Suicide Life Threat Behav. 2019;49(1):328-37.
  5. [8]8. Smith R, Taylor S, Bilek E. Computational mechanisms of addiction: recent evidence and its relevance to addiction medicine. Current Addiction Reports. 2021:(In Press).
  6. [9]9. Huys QJ, Maia TV, Frank MJ. Computational psychiatry as a bridge from neuroscience to clinical applications. Nat Neurosci. 2016;19(3):404-13.
  7. [10]10. Wilson R, Geana A, White J, Ludvig E, Cohen J. Humans use directed and random exploration to solve the explore-exploit dilemma. Journal of experimental psychology General. 2014;143:2074-81. . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted April 5, 2023. ; https://doi.org/10.1101/2023.04.03.23288037doi: medRxiv preprint https://doi.org/10.1101/2023.04.03.23288037 http://creativecommons.org/licenses/by-nc-nd/4.0/ MODELING LEARNING RATES IN SUBSTANCE USE 22 · DOI
  8. [11]11. Mollick JA, Kober H. Computational models of drug use and addiction: A review. J Abnorm Psychol. 2020;129(6):544-55.
  9. [12]12. Smith R, Taylor S, Bilek E. Computational mechanisms of addiction: recent evidence and its relevance to addiction medicine. Current Addiction Reports. 2021;8:509–19.
  10. [13]13. Valyan A, Ekhtiari H, Smith R, Paulus MP. Decision-making deficits in substance use disorders. In: Verdejo-Garcia A, editor. Cognition and Addiction: Academic Press; 2020. p. 25-
  11. [61]61. 14. Smith R, Kuplicki R, Feinstein J, Forthman KL, Stewart JL, Paulus MP, et al. A Bayesian computational model reveals a failure to adapt interoceptive precision estimates across depression, anxiety, eating, and substance use disorders. PLoS Computational Biology. 2020;16(12):e1008484.
  12. [15]15. Smith R, Feinstein JS, Kuplicki R, Forthman KL, Stewart JL, Paulus MP, et al. Perceptual insensitivity to the modulation of interoceptive signals in depression, anxiety, and substance use disorders. Sci Rep. 2021;11(1):2108.
  13. [16]16. Myers CE, Sheynin J, Balsdon T, Luzardo A, Beck KD, Hogarth L, et al. Probabilistic reward- and punishment-based learning in opioid addiction: Experimental and computational data. Behav Brain Res. 2016;296:240-8.
  14. [17]17. Kanen JW, Ersche KD, Fineberg NA, Robbins TW, Cardinal RN. Computational modelling reveals contrasting effects on reinforcement learning and cognitive flexibility in stimulant use disorder and obsessive-compulsive disorder: remediating effects of dopaminergic D2/3 receptor agents. Psychopharmacology (Berl). 2019;236(8):2337-58.
  15. [18]18. Konova AB, Lopez-Guzman S, Urmanche A, Ross S, Louie K, Rotrosen J, et al. Computational Markers of Risky Decision-making for Identification of Temporal Windows of Vulnerability to Opioid Use in a Real-world Clinical Setting. JAMA Psychiatry. 2020;77(4):368-77.
  16. [19]19. Smith R, Schwartenbeck P, Stewart JL, Kuplicki R, Ekhtiari H, Investigators T, et al. Imprecise Action Selection in Substance Use Disorder: Evidence for Active Learning Impairments When Solving the Explore-exploit Dilemma. Drug and Alcohol Dependence. 2020;215:108208.
  17. [20]20. Smith R, Taylor S, Stewart JL, Guinjoan SM, Ironside M, Kirlic N, et al. Slower Learning Rates from Negative Outcomes in Substance Use Disorder over a 1-Year Period and Their Potential Predictive Utility. Computational Psychiatry. 2022;6(1):117-41.
  18. [21]21. Victor TA, Khalsa SS, Simmons WK, Feinstein JS, Savitz J, Aupperle RL, et al. Tulsa 1000: a naturalistic study protocol for multilevel assessment and outcome prediction in a large psychiatric sample. BMJ open. 2018;8(1):e016620.
  19. [24]24. Bohn M, Babor T, Kranzler H. Validity of the Drug Abuse Screening Test (DAST-10) in inpatient substance abusers. Problems of drug dependence. 1991;119:233-5.
  20. [26]26. Morey RD, Rouder JN. BayesFactor (Version 0.9.10-2)[Computer software]. 2015. . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted April 5, 2023. ; https://doi.org/10.1101/2023.04.03.23288037doi: medRxiv preprint https://doi.org/10.1101/2023.04.03.23288037 http://creativecommons.org/licenses/by-nc-nd/4.0/ MODELING LEARNING RATES IN SUBSTANCE USE 23 · DOI
  21. [27]27. Johnstone B, Callahan CD, Kapila CJ, Bouman DE. The comparability of the WRAT-R reading test and NAART as estimates of premorbid intelligence in neurologically impaired patients. Arch Clin Neuropsychol. 1996;11(6):513-9.
  22. [28]28. Zhang S, Yu AJ. Forgetful Bayes and myopic planning: Human learning and decision- making in a bandit setting. Advances in neural information processing systems. 2013:2607-15.
  23. [31]31. Friston K, Mattout J, Trujillo-Barreto N, Ashburner J, Penny W. Variational free energy and the Laplace approximation. Neuroimage. 2007;34(1):220-34.
  24. [33]33. Rigoux L, Stephan KE, Friston KJ, Daunizeau J. Bayesian model selection for group studies - revisited. Neuroimage. 2014;84:971-85.
  25. [34]34. Stephan KE, Penny WD, Daunizeau J, Moran RJ, Friston KJ. Bayesian model selection for group studies. Neuroimage. 2009;46(4):1004-17.
  26. [35]35. Grubbs F. Procedures for detecting outlying observations in samples. Technometrics. 1969;11(1):1-21.
  27. [36]36. Friston KJ, Litvak V, Oswal A, Razi A, Stephan KE, van Wijk BCM, et al. Bayesian model reduction and empirical Bayes for group (DCM) studies. Neuroimage. 2016;128:413-31.
  28. [37]37. Hosmer Jr. DW, Lemeshow S, Sturdivant RS. Applied Logistic Regression, Third Edition: John Wiley & Sons, Inc.; 2013.
  29. [38]38. Smith R, Kirlic N, Stewart JL, Touthang J, Kuplicki R, Khalsa SS, et al. Greater decision uncertainty characterizes a transdiagnostic patient sample during approach-avoidance conflict: a computational modelling approach. Journal of Psychiatry & Neuroscience. 2021;46(1):E74- E87.
  30. [39]39. Smith R, Kirlic N, Stewart JL, Touthang J, Kuplicki R, McDermott TJ, et al. Long-term stability of computational parameters during approach-avoidance conflict in a transdiagnostic psychiatric patient sample. Scientific Reports. 2021;11(1):11783.
  31. [40]40. Genauck A, Quester S, Wustenberg T, Morsen C, Heinz A, Romanczuk-Seiferth N. Reduced loss aversion in pathological gambling and alcohol dependence is associated with differential alterations in amygdala and prefrontal functioning. Sci Rep. 2017;7(1):16306.
  32. [41]41. Fridberg DJ, Queller S, Ahn WY, Kim W, Bishara AJ, Busemeyer JR, et al. Cognitive Mechanisms Underlying Risky Decision-Making in Chronic Cannabis Users. J Math Psychol. 2010;54(1):28-38.
  33. [42]42. Ahn WY, Vasilev G, Lee SH, Busemeyer JR, Kruschke JK, Bechara A, et al. Decision-making in stimulant and opiate addicts in protracted abstinence: evidence from computational modeling with pure users. Front Psychol. 2014;5:849.
  34. [43]43. Lopez-Guzman S, Konova AB, Glimcher PW. Computational psychiatry of impulsivity and risk: how risk and time preferences interact in health and disease. Philos Trans R Soc Lond B Biol Sci. 2019;374(1766):20180135. . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted April 5, 2023. ; https://doi.org/10.1101/2023.04.03.23288037doi: medRxiv preprint https://doi.org/10.1101/2023.04.03.23288037 http://creativecommons.org/licenses/by-nc-nd/4.0/ MODELING LEARNING RATES IN SUBSTANCE USE 24 · DOI
  35. [44]44. Beylergil SB, Beck A, Deserno L, Lorenz RC, Rapp MA, Schlagenhauf F, et al. Dorsolateral prefrontal cortex contributes to the impaired behavioral adaptation in alcohol dependence. Neuroimage Clin. 2017;15:80-94.
  36. [45]45. Gowin JL, Mackey S, Paulus MP. Altered risk-related processing in substance users: imbalance of pain and gain. Drug Alcohol Depend. 2013;132(1-2):13-21.
  37. [46]46. Simons JS, Dvorak RD, Batien BD. Methamphetamine use in a rural college population: associations with marijuana use, sensitivity to punishment, and sensitivity to reward. Psychol Addict Behav. 2008;22(3):444-9.
  38. [47]47. Simons JS, Arens AM. Moderating effects of sensitivity to punishment and sensitivity to reward on associations between marijuana effect expectancies and use. Psychol Addict Behav. 2007;21(3):409-14.
  39. [48]48. Stewart JL, May AC, Poppa T, Davenport PW, Tapert SF, Paulus MP. You are the danger: attenuated insula response in methamphetamine users during aversive interoceptive decision- making. Drug Alcohol Depend. 2014;142:110-9.
  40. [49]49. Hester R, Bell RP, Foxe JJ, Garavan H. The influence of monetary punishment on cognitive control in abstinent cocaine-users. Drug Alcohol Depend. 2013;133(1):86-93. . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted April 5, 2023. ; https://doi.org/10.1101/2023.04.03.23288037doi: medRxiv preprint https://doi.org/10.1101/2023.04.03.23288037 http://creativecommons.org/licenses/by-nc-nd/4.0/ MODELING LEARNING RATES IN SUBSTANCE USE 25 · DOI

Cited by

Sorted by citations

Authors
Taylor, S., Lavalley, C. A., Hakimi, N., Stewart, J. L., Ironside, M., Zheng, H., White, E., Guinjoan, S., Paulus, M. P., & Smith, R.
Affiliations
Laureate Institute for Brain Research (US)
Corresponding author
Smith, Ryan
ORCID
Venue
Drug and Alcohol Dependence
Publisher
Elsevier BV
Published
2023-11
Volume
252
Pages
110945
ISSN
0376-8716 (ISSN-L)
DOI
10.1016/j.drugalcdep.2023.110945
PMC
PMC10635739
OpenAlex
W4386171629
Cited by
15
References
7
Citations by year
2026: 3 · 2025: 5 · 2024: 4 · 2023: 3
FWCI
3.00
Type
journal article
Language
en
Open access
Licence
Locations
  • Drug and Alcohol Dependence (acceptedVersion)
  • medRxiv (acceptedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • PubMed Central (submittedVersion, cc-by-nc-nd)
Funders
  • Laureate Institute for Brain Research, University of Tulsa
  • National Institute of General Medical Sciences (P20 GM121312)
Topics
  • Health Sciences › Medicine › Epidemiology › Substance Abuse Treatment and Outcomes
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Social Sciences › Psychology › Clinical Psychology › Gambling Behavior and Treatments
Keywords
substance use disorders, active learning (machine learning), learning rate, computational modeling, replication, transdiagnostic mechanisms, treatment goals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review