The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Modeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning
Open accessPublished version of a preprint
Abstract
BACKGROUND: Serious and debilitating symptoms of anxiety are the most common mental health problem worldwide, accounting for around 5% of all adult years lived with disability in the developed world. Avoidance behavior-avoiding social situations for fear of embarrassment, for instance-is a core feature of such anxiety. However, as for many other psychiatric symptoms the biological mechanisms underlying avoidance remain unclear. METHODS: Reinforcement learning models provide formal and testable characterizations of the mechanisms of decision making; here, we examine avoidance in these terms. A total of 101 healthy participants and individuals with mood and anxiety disorders completed an approach-avoidance go/no-go task under stress induced by threat of unpredictable shock. RESULTS: We show an increased reliance in the mood and anxiety group on a parameter of our reinforcement learning model that characterizes a prepotent (pavlovian) bias to withhold responding in the face of negative outcomes. This was particularly the case when the mood and anxiety group was under stress. CONCLUSIONS: This formal description of avoidance within the reinforcement learning framework provides a new means of linking clinical symptoms with biophysically plausible models of neural circuitry and, as such, takes us closer to a mechanistic understanding of mood and anxiety disorders.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Aversion models in rodents are perhaps the closest pre-clinical approximation to avoidance behaviors [56], although limitations regarding clinical validity need to be kept in mind when interpreting results.”
From Baseline Pro-Inflammatory Cytokine Levels Moderate Psychological Inflexibility in Behavioral Treatment for Chronic Pain (2022) · cites this paper as [56]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“For example, Mkrtchian et al [18] recently demonstrated avoidance as a part of reinforcement learning pathways in individuals with anxiety disorders.”
From Clinical Efficacy and Psychological Mechanisms of an App-Based Digital Therapeutic for Generalized Anxiety Disorder: Randomized Controlled Trial (2021) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 7,268
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Sheehan, D., Lecrubier, Y., Harnett Sheehan, K., Janavs, J., Weiller, E., Keskiner, A., Schinka, J., Knapp, E., Sheehan, M., & Dunbar, G. (1997) · European Psychiatry · doi:10.1016/s0924-9338(97)83297-x · Cited by 2,149
Monroe, S. M., & Simons, A. D. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.3.406 · Cited by 2,143
Phasic vs Sustained Fear in Rats and Humans: Role of the Extended Amygdala in Fear vs Anxiety
Davis, M., Walker, D. L., Miles, L., & Grillon, C. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.109 · Cited by 1,513 · cited 1×
“, comparable behavioral responses can be elicited across human and animal models (5)] threat-ofshock procedure to induce stress (6,7); 2) a cognitive task that has been shown to reliably index avoidance behavior in healthy individuals (1); and 3) a computationally precise method of defining of avoidance.”
Computational psychiatry as a bridge from neuroscience to clinical applications
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4238 · Cited by 1,002
The impact of anxiety upon cognition: perspectives from human threat of shock studies
Robinson, O. J., Vytal, K., Cornwell, B. R., & Grillon, C. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00203 · Cited by 613
Self-citationGo and no-go learning in reward and punishment: Interactions between affect and effect
Guitart‐Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Düzel, E., & Dolan, R. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.024 · Cited by 448
Self-citationDisentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding
Huys, Q. J. M., Cools, R., Gölzer, M., Friedel, E., Heinz, A., Dolan, R. J., & Dayan, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002028 · Cited by 436
Self-citationA computational model of inhibitory control in frontal cortex and basal ganglia.
Wiecki, T. V., & Frank, M. J. (2013) · Psychological Review · doi:10.1037/a0031542 · Cited by 423
Avoidance learning: a review of theoretical models and recent developments
Krypotos, A. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2015.00189 · Cited by 402
Ahn, W., Haines, N., & Zhang, L. (2017) · Computational Psychiatry · doi:10.1162/cpsy_a_00002 · Cited by 385
Schmitz, A., & Grillon, C. (2012) · Nature Protocols · doi:10.1038/nprot.2012.001 · Cited by 383
The birth, death and resurrection of avoidance: a reconceptualization of a troubled paradigm
LeDoux, J. E., Moscarello, J., Sears, R., & Campese, V (2017) · Molecular Psychiatry · doi:10.1038/mp.2016.166 · Cited by 348
Crockett, M. J., Clark, L., & Robbins, T. W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2513-09.2009 · Cited by 271
Dispositional negativity: An integrative psychological and neurobiological perspective.
Shackman, A. J., Tromp, D., Stockbridge, M. D., Kaplan, C. M., Tillman, R. M., & Fox, A. S. (2016) · Psychological Bulletin · doi:10.1037/bul0000073 · Cited by 249
Extinction of avoidance responding through response prevention (flooding).
Baum, M. (1970) · Psychological Bulletin · doi:10.1037/h0029789 · Cited by 233
Robinson, O. J., Cools, R., Carlisi, C. O., Sahakian, B. J., & Drevets, W. C. (2012) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11010137 · Cited by 232
Self-citationGuitart-Masip, M., Fuentemilla, L., Bach, D. R., Huys, Q. J. M., Dayan, P., Dolan, R. J., & Duzel, E. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6376-10.2011 · Cited by 216
Self-citationReward fails to alter response bias in depression.
Henriques, J. B., Glowacki, J. M., & Davidson, R. J. (1994) · Journal of Abnormal Psychology · doi:10.1037//0021-843x.103.3.460 · Cited by 215
Reward fails to alter response bias in depression.
Henriques, J. B., Glowacki, J. M., & Davidson, R. J. (1994) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.103.3.460 · Cited by 172
An approach through the looking-glass
Hershberger, W. A. (1986) · Animal Learning & Behavior · doi:10.3758/bf03200092 · Cited by 149
Forster, S., Núñez-Elizalde, A. O., Castle, E., & Bishop, S. J. (2013) · Cerebral Cortex · doi:10.1093/cercor/bht248 · Cited by 143
Action controls dopaminergic enhancement of reward representations
Guitart‐Masip, M., Chowdhury, R., Sharot, T., Dayan, P., Düzel, E., & Dolan, R. J. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1202229109 · Cited by 127
Self-citationGuitart‐Masip, M., Economides, M., Huys, Q. J. M., Frank, M. J., Chowdhury, R., Düzel, E., Dayan, P., & Dolan, R. J. (2013) · Psychopharmacology · doi:10.1007/s00213-013-3313-4 · Cited by 113
Self-citationThe impact of induced anxiety on response inhibition
Robinson, O. J., Krimsky, M., & Grillon, C. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00069 · Cited by 108
Self-citationStress increases aversive prediction error signal in the ventral striatum
Robinson, O. J., Overstreet, C. M., Charney, D. R., Vytal, K. E., & Grillon, C. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1213923110 · Cited by 97
Self-citationStriatal structure and function predict individual biases in learning to avoid pain
Eldar, E., Hauser, T. U., Dayan, P., & Dolan, R. J. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1519829113 · Cited by 79
Self-citationAhn, W., Haines, N., & Zhang, L. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/064287 · Cited by 67
Anxiety-Like Behavioural Inhibition Is Normative under Environmental Threat-Reward Correlations
Bach, D. R. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004646 · Cited by 62
Clinical anxiety promotes excessive response inhibition
Grillon, C., Robinson, O. J., O'Connell, K., Davis, A., Alvarez, G., Pine, D. S., & Ernst, M. (2017) · Psychological Medicine · doi:10.1017/s0033291716002555 · Cited by 56
Self-citationThe Effects of Approach–Avoidance Modification on Social Anxiety Disorder: A Pilot Study
Asnaani, A., Rinck, M., Becker, E. S., & Hofmann, S. G. (2013) · Cognitive Therapy and Research · doi:10.1007/s10608-013-9580-x · Cited by 52
The effect of modifying automatic action tendencies on overt avoidance behaviors.
Amir, N., Kuckertz, J. M., & Najmi, S. (2013) · Emotion · doi:10.1037/a0030443 · Cited by 44
Olff, M., Sijbrandij, M., Opmeer, B. C., Carlier, I. V. E., & Gersons, B. P. R. (2009) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2009.02.003 · Cited by 36
Grillon, C., Robinson, O. J., Krimsky, M., O’Connell, K., Alvarez, G., & Ernst, M. (2016) · Emotion · doi:10.1037/emo0000214 · Cited by 29
Self-citationWhat has neuroscience ever done for us
Roiser, J. P. (2015) · UCL Discovery (University College London) · Cited by 12
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Penninx, B. W., Pine, D. S., Holmes, E. A., & Reif, A. (2021) · The Lancet · doi:10.1016/s0140-6736(21)00359-7 · Cited by 740
The prefrontal cortex, pathological anxiety, and anxiety disorders
Kenwood, M. M., Kalin, N. H., & Barbas, H. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01109-z · Cited by 379
Chavanne, A. V., & Robinson, O. J. (2020) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2020.19111153 · Cited by 249
Self-citationEnhanced Risk Aversion, But Not Loss Aversion, in Unmedicated Pathological Anxiety
Charpentier, C. J., Aylward, J., Roiser, J. P., & Robinson, O. J. (2016) · Biological Psychiatry · doi:10.1016/j.biopsych.2016.12.010 · Cited by 210
Self-citationAdvances in the computational understanding of mental illness
Huys, Q. J. M., Browning, M. D., Paulus, M. P., & Frank, M. J. (2020) · Neuropsychopharmacology · doi:10.1038/s41386-020-0746-4 · Cited by 204
Metadata
- Authors
- Mkrtchian, A., Aylward, J., Dayan, P., Roiser, J. P., & Robinson, O. J.
- Affiliations
- University College London (GB)
- Gatsby Computational Neuroscience Unit (GB)
- Corresponding author
- Robinson, Oliver J.
- Venue
- Biological Psychiatry
- Publisher
- Elsevier BV
- Published
- 2017-10
- Volume
- 82
- Issue
- 7
- Pages
- 532-539
- ISSN
- 0006-3223 (ISSN-L)
- PMC
- PMC5598542
- OpenAlex
- W2587263457
- Cited by
- 151
- References
- 35
- Citations by year
- 2026: 16 · 2025: 20 · 2024: 13 · 2023: 25 · 2022: 13 · 2021: 23 · 2020: 14 · 2019: 15 · 2018: 7 · 2017: 4
- FWCI
- 14.24
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Biological Psychiatry (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- Medical Research Council (MR/K024280/1)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- avoidance behavior, mood disorders, anxiety disorders, reinforcement learning models, go/no-go task, stress, Pavlovian bias, neural circuits
- MeSH terms
- Adolescent, Adult, Anxiety, Avoidance Learning, Bayes Theorem, Decision Making, Female, Humans, Male, Middle Aged, Models, Psychological, Psychiatric Status Rating Scales, Reinforcement, Psychology, Mood Disorders, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
