A Symbiotic Brain-Machine Interface through Value-Based Decision Making
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
BACKGROUND: In the development of Brain Machine Interfaces (BMIs), there is a great need to enable users to interact with changing environments during the activities of daily life. It is expected that the number and scope of the learning tasks encountered during interaction with the environment as well as the pattern of brain activity will vary over time. These conditions, in addition to neural reorganization, pose a challenge to decoding neural commands for BMIs. We have developed a new BMI framework in which a computational agent symbiotically decoded users' intended actions by utilizing both motor commands and goal information directly from the brain through a continuous Perception-Action-Reward Cycle (PARC). METHODOLOGY: The control architecture designed was based on Actor-Critic learning, which is a PARC-based reinforcement learning method. Our neurophysiology studies in rat models suggested that Nucleus Accumbens (NAcc) contained a rich representation of goal information in terms of predicting the probability of earning reward and it could be translated into an evaluative feedback for adaptation of the decoder with high precision. Simulated neural control experiments showed that the system was able to maintain high performance in decoding neural motor commands during novel tasks or in the presence of reorganization in the neural input. We then implanted a dual micro-wire array in the primary motor cortex (M1) and the NAcc of rat brain and implemented a full closed-loop system in which robot actions were decoded from the single unit activity in M1 based on an evaluative feedback that was estimated from NAcc. CONCLUSIONS: Our results suggest that adapting the BMI decoder with an evaluative feedback that is directly extracted from the brain is a possible solution to the problem of operating BMIs in changing environments with dynamic neural signals. During closed-loop control, the agent was able to solve a reaching task by capturing the action and reward interdependency in the brain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Simple model of spiking neurons
Izhikevich, E. M. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.820440 · Cited by 4,828
Neuronal ensemble control of prosthetic devices by a human with tetraplegia
Hochberg, L. R., Serruya, M. D., Friehs, G. M., Mukand, J. A., Saleh, M., Caplan, A. H., Branner, A., Chen, D., Penn, R. D., & Donoghue, J. P. (2006) · Nature · doi:10.1038/nature04970 · Cited by 3,295 · cited 1×
“While great progress has been made in BMI design, current approaches for decoding neural activity into behavior completely ignore the major aspects of the PARC by relying heavily only on the primary motor representation of behavior”
MethodFrom motivation to action: Functional interface between the limbic system and the motor system
MOGENSON, G., JONES, D., & YIM, C. (1980) · Progress in Neurobiology · doi:10.1016/0301-0082(80)90018-0 · Cited by 2,331 · cited 1×
“Integration of reward perception and motor information in the NAcc has given rise to the idea that NAcc serves as limbic-motor interface [26].”
BackgroundA framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184 · cited 3×
“These approaches solve functional regression problems but are devoid of the key components that are known to play a role in goal-directed action selection [6].”
BackgroundInfluentialCORTICAL PLASTICITY: From Synapses to Maps
Buonomano, D. V., & Merzenich, M. M. (1998) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.21.1.149 · Cited by 2,083 · cited 1×
“Brain plasticity is an important design factor of BMI and it has been observed in the context of many research areas [47,48].”
BackgroundBrain–machine interfaces: past, present and future
Lebedev, M. A., & Nicolelis, M. A. L. (2006) · Trends in Neurosciences · doi:10.1016/j.tins.2006.07.004 · Cited by 1,875 · cited 1×
“. For BMIs, the acquisition of goal-directed behaviors is critical for reach to grasp motions, which are highly desirable for paralyzed patients”
BackgroundLearning to control a brain–machine interface for reaching and grasping by primates
Carmena, J. M., Lebedev, M. A., Crist, R. E., O'Doherty, J. E., Santucci, D. M., Dimitrov, D. F., Patil, P. G., Henriquez, C. S., & Nicolelis, M. A. L. (2003) · PLoS Biology · doi:10.1371/journal.pbio.0000042 · Cited by 1,761
Cortical control of a prosthetic arm for self-feeding
Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S., & Schwartz, A. B. (2008) · Nature · doi:10.1038/nature06996 · Cited by 1,591
Multiple reward signals in the brain
Schultz, W. (2000) · Nature Reviews Neuroscience · doi:10.1038/35044563 · Cited by 1,303 · cited 1×
“The encoding of goal-directed, rewarding behavior has been localized to many centers in the brain [19].”
BackgroundBiological substrates of reward and aversion: A nucleus accumbens activity hypothesis
Carlezon, W. A., & Thomas, M. J. (2009) · Neuropharmacology · doi:10.1016/j.neuropharm.2008.06.075 · Cited by 561
Roitman, M. F., Wheeler, R. A., & Carelli, R. M. (2005) · Neuron · doi:10.1016/j.neuron.2004.12.055 · Cited by 453 · cited 1×
“Depending upon the user’s goal, IA actions might increase or decrease the reward expectation [33].”
BackgroundUpper processing stages of the perception–action cycle
Fuster, J. M. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.02.004 · Cited by 380 · cited 1×
“By engaging a Perception-Action-Reward Cycle (PARC) through an intricate set of sensorimotor processes [2,3] internal antecedents are expressed through actions and ultimately the outcomes of these actions contribute to shaping future motor behaviors [4,5].”
BackgroundDifferential Corticostriatal Plasticity during Fast and Slow Motor Skill Learning in Mice
Costa, R. M., Cohen, D., & Nicolelis, M. A. L. (2004) · Current Biology · doi:10.1016/j.cub.2004.06.053 · Cited by 373 · cited 1×
“Brain plasticity is an important design factor of BMI and it has been observed in the context of many research areas”
BackgroundReward prediction in primate basal ganglia and frontal cortex
Schultz, W., Tremblay, L., & Hollerman, J. R. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00071-9 · Cited by 318 · cited 1×
“In this regard, the striatum is a key structure that represents action-specific reward values in cortico-basal ganglia loops [22,23] and encodes reward expectation of actions [24] during goal-directed behavior.”
Background
Cited by
Applications of deep learning and reinforcement learning to biological data
Mahmud, M., Kaiser, M. S., Hussain, A., & Vassanelli, S. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2790388 · Cited by 765
Restoring sensorimotor function through intracortical interfaces: progress and looming challenges
Bensmaia, S. J., & Miller, L. E. (2014) · Nature Reviews Neuroscience · doi:10.1038/nrn3724 · Cited by 368
Teaching brain-machine interfaces as an alternative paradigm to neuroprosthetics control
Iturrate, I., Chavarriaga, R., Montesano, L., Minguez, J., & Millán, J. D. R. (2015) · Scientific Reports · doi:10.1038/srep13893 · Cited by 149
Advances in Neuroprosthetic Learning and Control
Carmena, J. M. (2013) · PLoS Biology · doi:10.1371/journal.pbio.1001561 · Cited by 117
“The error signals required to adapt the decoder can be estimated in a variety of different ways, including using the task goals to infer the subject’s intention [33], Bayesian methods to self-train the decoder [34], and extracting error signals directly from the brain [35].”
MethodDesign and analysis of closed-loop decoder adaptation algorithms for brain-machine interfaces
Dangi, S., Orsborn, A. L., Moorman, H. G., & Carmena, J. M. (2013) · Neural Computation · doi:10.1162/neco_a_00460 · Cited by 103
Metadata
- Authors
- Mahmoudi, B., & Sanchez, J. C.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-03-14
- Volume
- 6
- Issue
- 3
- Pages
- e14760
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2076114331
- Cited by
- 88
- References
- 15
- FWCI
- 5.73
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.