The DOI matches its Crossref record; 114 of the 114 references it cites were found in a registry or an index.
Disentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding
Open accessAbstract
Hard-wired, Pavlovian, responses elicited by predictions of rewards and punishments exert significant benevolent and malevolent influences over instrumentally-appropriate actions. These influences come in two main groups, defined along anatomical, pharmacological, behavioural and functional lines. Investigations of the influences have so far concentrated on the groups as a whole; here we take the critical step of looking inside each group, using a detailed reinforcement learning model to distinguish effects to do with value, specific actions, and general activation or inhibition. We show a high degree of sophistication in Pavlovian influences, with appetitive Pavlovian stimuli specifically promoting approach and inhibiting withdrawal, and aversive Pavlovian stimuli promoting withdrawal and inhibiting approach. These influences account for differences in the instrumental performance of approach and withdrawal behaviours. Finally, although losses are as informative as gains, we find that subjects neglect losses in their instrumental learning. Our findings argue for a view of the Pavlovian system as a constraint or prior, facilitating learning by alleviating computational costs that come with increased flexibility.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hamilton, M. (1960) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.23.1.56 · Cited by 32,817 · cited 1×
“After completing the tasks, subjects completed self-rating scales (Beck Depression Inventory II (BDI), Beck Anxiety Inventory (BAI), State-Trait Anxiety Inventory STAI [103–105]), followed by the administration of clinician rated scales (Montgomery-Ashberg Depression Rating Scale (MADRS), Hamilton Depression Scale (Ha…”
MethodReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
An inventory for measuring clinical anxiety: Psychometric properties.
Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.56.6.893 · Cited by 14,407
A new depression scale designed to be sensitive to change
Montgomery, S. A., & Åsberg, M. (1979) · British Journal of Psychiatry · doi:10.1192/bjp.134.4.382 · Cited by 13,962
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
Information theory, inference, and learning algorithms
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5949 · Cited by 6,362
Functional architecture of basal ganglia circuits: neural substrates of parallel processing
Alexander, G. E., & Crutcher, M. D. (1990) · Trends in Neurosciences · doi:10.1016/0166-2236(90)90107-l · Cited by 4,131
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718
Experimental Tests of the Endowment Effect and the Coase Theorem
Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1990) · Journal of Political Economy · doi:10.1086/261737 · Cited by 3,194 · cited 2×
“Pavlovian interference likely contributes to many quirks of behaviour such as impulsivity [12], framing and [13], endowment effects [14] and many other ‘‘anomalies’’ [15], including neurological [16–19] and psychiatric diseases [20–26].”
Background
Cited by
Ten simple rules for the computational modeling of behavioral data
Wilson, R. C., & Collins, A. G. (2019) · eLife · doi:10.7554/elife.49547 · Cited by 710
Characterizing a psychiatric symptom dimension related to deficits in goal-directed control
Gillan, C. M., Kosinski, M., Whelan, R., Phelps, E. A., & Daw, N. D. (2016) · eLife · doi:10.7554/elife.11305 · Cited by 586
Barrett, L. F. (2012) · Emotion · doi:10.1037/a0027555 · Cited by 503
“Brain circuitry for approach and avoidance have been identified in both human (e.g., Huys et al., 2011) and nonhuman animals (e.g., Ferry & McGaugh, 2008; Ikemoto, 2010; Numan & Insel, 2003).”
BackgroundMapping anhedonia onto reinforcement learning: a behavioural meta-analysis
Huys, Q. J., Pizzagalli, D. A., Bogdan, R., & Dayan, P. (2013) · Biology of Mood & Anxiety Disorders · doi:10.1186/2045-5380-3-12 · Cited by 480
“Note, though, that the current design cannot completely disentangle instrumental effects from those due to Pavlovian approach [58,99].”
BackgroundMethodInfluentialSelf-citationHuys, Q. J. M., Eshel, N., O'Nions, E., Sheridan, L., Dayan, P., & Roiser, J. P. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002410 · Cited by 439
Self-citationEmotion and Decision Making: Multiple Modulatory Neural Circuits
Phelps, E. A., Lempert, K. M., & Sokol-Hessner, P. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071013-014119 · Cited by 430
Go and no-go learning in reward and punishment: Interactions between affect and effect
Guitart-Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Duzel, E., & Dolan, R. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.024 · Cited by 421
Self-citationCollins, A. G. E., & Frank, M. J. (2014) · Psychological Review · doi:10.1037/a0037015 · Cited by 403
The successor representation in human reinforcement learning
Momennejad, I., Russek, E. M., Cheong, J. H., Botvinick, M. M., Daw, N. D., & Gershman, S. J. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0180-8 · Cited by 390
Ahn, W.-Y., Haines, N., & Zhang, L. (2017) · Computational Psychiatry · doi:10.1162/cpsy_a_00002 · Cited by 362
Appetitive Pavlovian-instrumental Transfer: A review
Cartoni, E., Balleine, B., & Baldassarre, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.020 · Cited by 317
Action versus valence in decision making
Guitart-Masip, M., Duzel, E., Dolan, R., & Dayan, P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.01.003 · Cited by 270
Self-citationDopamine restores reward prediction errors in old age
Chowdhury, R., Guitart-Masip, M., Lambert, C., Dayan, P., Huys, Q., Düzel, E., & Dolan, R. J. (2013) · Nature Neuroscience · doi:10.1038/nn.3364 · Cited by 273
Self-citationCrockett, M. J. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.06.005 · Cited by 264
Stratified medicine for mental disorders
Schumann, G., Binder, E. B., Holte, A., de Kloet, E. R., Oedegaard, K. J., Robbins, T. W., Walker-Tilley, T. R., Bitter, I., Brown, V. J., Buitelaar, J., et al. (2014) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2013.09.010 · Cited by 186
Self-citationThe Anterior Cingulate Cortex Predicts Future States to Mediate Model-Based Action Selection
Akam, T., Rodrigues-Vaz, I., Marcelo, I., Zhang, X., Pereira, M., Oliveira, R. F., Dayan, P., & Costa, R. M. (2021) · Neuron · doi:10.1016/j.neuron.2020.10.013 · Cited by 170
Self-citationModeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning
Mkrtchian, A., Aylward, J., Dayan, P., Roiser, J. P., & Robinson, O. J. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.01.017 · Cited by 151
“During fitting, parameters are constrained to within meaningful ranges [see (12)].”
InfluentialSelf-citationModel averaging, optimal inference, and habit formation
FitzGerald, T. H. B., Dolan, R. J., & Friston, K. J. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00457 · Cited by 121
Self-citationIigaya, K., Fonseca, M. S., Murakami, M., Mainen, Z. F., & Dayan, P. (2018) · Nature Communications · doi:10.1038/s41467-018-04840-2 · Cited by 120
Self-citationBerwian, I. M., Wenzel, J. G., Collins, A. G. E., Seifritz, E., Stephan, K. E., Walter, H., & Huys, Q. J. M. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.4971 · Cited by 113
Self-citationFolloni, D., Fouragnan, E., Wittmann, M. K., Roumazeilles, L., Tankelevitch, L., Verhagen, L., Attali, D., Aubry, J.-F., Sallet, J., & Rushworth, M. F. S. (2021) · Science Advances · doi:10.1126/sciadv.abg7700 · Cited by 75
Approach-Induced Biases in Human Information Sampling
Hunt, L. T., Rutledge, R. B., Malalasekera, W. M. N., Kennerley, S. W., & Dolan, R. J. (2016) · PLOS Biology · doi:10.1371/journal.pbio.2000638 · Cited by 60
Self-citationSerotonin modulates asymmetric learning from reward and punishment in healthy human volunteers
Michely, J., Eldar, E., Erdman, A., Martin, I. M., & Dolan, R. J. (2022) · Communications Biology · doi:10.1038/s42003-022-03690-5 · Cited by 43
Self-citation
Metadata
- Authors
- Huys, Q. J. M., Cools, R., Gölzer, M., Friedel, E., Heinz, A., Dolan, R. J., & Dayan, P.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-04-21
- Volume
- 7
- Issue
- 4
- Pages
- e1002028
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2083607057
- Cited by
- 416
- References
- 114
- FWCI
- 8.44
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.