Direct Instrumental Conditioning of Neural Activity Using Functional Magnetic Resonance Imaging-Derived Reward Feedback
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Successful learning is often contingent on feedback. In instrumental conditioning, an animal or human learns to perform specific responses to obtain reward. Instrumental conditioning is often used by behavioral psychologists to train an animal (or human) to produce a desired behavior. Shaping involves reinforcing those behaviors, which in a stepwise manner are successively closer to the desired behavior until the desired behavior is reached. Here, we aimed to extend this traditional approach to directly shape neural activity instead of overt behavior. To achieve this, we scanned 22 human subjects with functional magnetic resonance imaging and performed image processing in parallel with acquisition. We delineated regions of interest (ROIs) in finger and toe motor/somatosensory regions and used an instrumental shaping procedure to induce a regionally specific increase in activity by providing an explicit monetary reward to reinforce neural activity in the target areas. After training, we found a significant and regionally specific increase in activity in the ROI being rewarded (finger or toe) and a decrease in activity in the nonrewarded region. This demonstrates that instrumental conditioning procedures can be used to directly shape neural activity, even without the production of an overt behavioral response. This procedure offers an important alternative to traditional biofeedback-based approaches and may be useful in the development of future therapies for stroke and other brain disorders.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data
Eickhoff, S. B., Stephan, K. E., Mohlberg, H., Grefkes, C., Fink, G. R., Amunts, K., & Zilles, K. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.12.034 · Cited by 4,033 · cited 2×
“941] located on the left precentral gyrus (Broadmann areas 4a, 6, and 1) (Eickhoff et al., 2005); individual-subject ROI centers were located near the hand knob (Yousry et al.”
BackgroundMethodInfluentialGoal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541 · cited 2×
“Traditionally, the response consists of overt behavioral actions, such as pulling a lever, traversing a maze, or pressing a button (Small, 1901; Thorndike, 1911; Skinner, 1938; Ljungberg et al., 1992; Balleine and Dickinson, 1998).”
BackgroundTemporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Self-citationResponses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Representation of Action-Specific Reward Values in the Striatum
Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005) · Science · doi:10.1126/science.1115270 · Cited by 939 · cited 1×
“…conditioning means that we can benefit from the extensive work done in this area to inform our understanding of the neural and behavioral processes mediating this learning (Balleine and Dickinson, 1998; Gottfried et al., 2002; O’Doherty et al., 2003; Samejima et al., 2005; Yacubian et al., 2006).”
BackgroundPartially overlapping neural networks for real and imagined hand movements.
Gerardin, E. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.11.1093 · Cited by 930
Control over brain activation and pain learned by using real-time functional MRI
deCharms, R. C., Maeda, F., Glover, G. H., Ludlow, D., Pauly, J. M., Soneji, D., Gabrieli, J. D. E., & Mackey, S. C. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0505210102 · Cited by 876 · cited 4×
“, 2004) (for review, see Weiskopf et al., 2004; deCharms et al., 2005).”
MethodInfluentialCognitive Control Signals for Neural Prosthetics
Musallam, S., Corneil, B. D., Greger, B., Scherberger, H., & Andersen, R. A. (2004) · Science · doi:10.1126/science.1097938 · Cited by 663 · cited 2×
“Musallam et al. (2004) demonstrated that by recording from neurons in parietal cortex, monkeys could be trained to generate neural responses to obtain juice rewards, without emitting any behavior.”
BackgroundYacubian, J., Gläscher, J., Schroeder, K., Sommer, T., Braus, D. F., & Büchel, C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2915-06.2006 · Cited by 537
The thought translation device (TTD) for completely paralyzed patients
Birbaumer, N., Kubler, A., Ghanayim, N., Hinterberger, T., Perelmouter, J., Kaiser, J., Iversen, I., Kotchoubey, B., Neumann, N., & Flor, H. (2000) · IEEE Transactions on Rehabilitation Engineering · doi:10.1109/86.847812 · Cited by 493
Operant Conditioning of Gill Withdrawal in Aplysia
Hawkins, R. D., Clark, G. A., & Kandel, E. R. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.3294-05.2006 · Cited by 41
Cited by
Closed-loop brain training: the science of neurofeedback
Sitaram, R., Ros, T., Stoeckel, L., Haller, S., Scharnowski, F., Lewis-Peacock, J., Weiskopf, N., Blefari, M. L., Rana, M., Oblak, E., et al. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2016.164 · Cited by 1,306
Sensorimotor training in virtual reality: A review
Adamovich, S. V., Fluet, G. G., Tunik, E., & Merians, A. S. (2009) · NeuroRehabilitation · doi:10.3233/nre-2009-0497 · Cited by 507
Perceptual Learning Incepted by Decoded fMRI Neurofeedback Without Stimulus Presentation
Shibata, K., Watanabe, T., Sasaki, Y., & Kawato, M. (2011) · Science · doi:10.1126/science.1212003 · Cited by 463
Real-time fMRI neurofeedback: Progress and challenges
Sulzer, J., Haller, S., Scharnowski, F., Weiskopf, N., Birbaumer, N., Blefari, M. L., Bruehl, A. B., Cohen, L. G., deCharms, R. C., Gassert, R., et al. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.03.033 · Cited by 453
“First, one must determine the effects of rtfMRI neurofeedback in healthy participants (e.g. Bray et al., 2007; Shibata et al., 2011; Weiskopf et al., 2003).”
BackgroundMethodInfluentialApplications of real-time fMRI
Christopher deCharms, R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2414 · Cited by 370
Metadata
- Authors
- Bray, S., Shimojo, S., & O'Doherty, J. P.
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2007-07-11
- Volume
- 27
- Issue
- 28
- Pages
- 7498-7507
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- OpenAlex
- W2077703305
- Cited by
- 138
- References
- 11
- FWCI
- 3.42
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.