CoChat
Sign Up
PreprintCited by 19EEG and Brain-Computer InterfacesNeuroscience and Neural EngineeringGaze Tracking and Assistive Technology

Plug-and-Play Stability for Intracortical Brain-Computer Interfaces: A One-Year Demonstration of Seamless Brain-to-Text Communication

Open access
Fan, Chaofei, Hahn, Nick, Kamdar, Foram, Avansino, Donald, Wilson, Guy H., Hochberg, Leigh, Shenoy, Krishna V., Henderson, Jaimie M. +1 more·2023·arXiv
DOI 10.48550/arxiv.2311.03611arXiv 2311.03611PMC PMC10659441
Open this paper in CoChat — free accountOpen PDF

Abstract

Intracortical brain-computer interfaces (iBCIs) have shown promise for restoring rapid communication to people with neurological disorders such as amyotrophic lateral sclerosis (ALS). However, to maintain high performance over time, iBCIs typically need frequent recalibration to combat changes in the neural recordings that accrue over days. This requires iBCI users to stop using the iBCI and engage in supervised data collection, making the iBCI system hard to use. In this paper, we propose a method that enables self-recalibration of communication iBCIs without interrupting the user. Our method leverages large language models (LMs) to automatically correct errors in iBCI outputs. The self-recalibration process uses these corrected outputs ("pseudo-labels") to continually update the iBCI decoder online. Over a period of more than one year (403 days), we evaluated our Continual Online Recalibration with Pseudo-labels (CORP) framework with one clinical trial participant. CORP achieved a stable decoding accuracy of 93.84% in an online handwriting iBCI task, significantly outperforming other baseline methods. Notably, this is the longest-running iBCI stability demonstration involving a human participant. Our results provide the first evidence for long-term stabilization of a plug-and-play, high-performance communication iBCI, addressing a major barrier for the clinical translation of iBCIs.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2311.03611

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In the automatic recalibration approach, MEA activity is decoded into subword units, converted to sentences by language models, and the language model output is used as pseudo-labels to update the RNN decoder.

    “In this approach, MEA activity is decoded into subword units and converted into sentences using language models; this language model output is used as a ‘pseudo-label’ to update the RNN decoding model9,16,105.”

    From The speech neuroprosthesis (2024) · cites this paper as [105]

  • A text-decoding-specific approach uses large language models to automatically recalibrate MEA-based text neuroprostheses.

    “A second approach, specific to text decoding, involves leveraging large language models to automatically recalibrate MEA-based text neuroprostheses.”

    From The speech neuroprosthesis (2024) · cites this paper as [105]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 22

  1. Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019. · arXiv
  2. Fabio M Carlucci, Antonio D'Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2229-2238, 2019.
  3. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014. · arXiv
  4. Ali Farshchian, Juan A Gallego, Joseph P Cohen, Yoshua Bengio, Lee E Miller, and Sara A Solla. Adversarial domain adaptation for stable brain-machine interfaces. arXiv preprint arXiv:1810.00045, 2018. · arXiv
  5. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The Pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. · arXiv
  6. Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pages 369-376, 2006.
  7. Junxian He, Jiatao Gu, Jiajun Shen, and Marc'Aurelio Ranzato. Revisiting self-training for neural sequence generation. arXiv preprint arXiv:1909.13788, 2019. · arXiv
  8. Christina Heinze-Deml and Nicolai Meinshausen. Conditional variance penalties and domain shift robustness. arXiv preprint arXiv:1710.11469, 2017. · arXiv
  9. Tommy Hosman, Tsam Kiu Pun, Kapitonava, John D Simeral, and Leigh R Hochberg. Months-long high-performance fixed LSTM decoder for cursor control in human intracortical brain-computer interfaces. In 2023 11th International IEEE/EMBS Conference on Neural Engineering (NER). IEEE, 2023.
  10. Weihua Hu, Gang Niu, Issei Sato, and Masashi Sugiyama. Does distributionally robust supervised learning give robust classifiers? In International Conference on Machine Learning, pages 2029-2037. PMLR, 2018.
  11. Jacob Kahn, Ann Lee, and Awni Hannun. Self-training for end-to-end speech recognition. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7084-7088. IEEE, 2020.
  12. Brianna M Karpowicz, Yahia H Ali, Lahiru N Wimalasena, Andrew R Sedler, Mohammad Reza Keshtkaran, Kevin Bodkin, Xuan Ma, Lee E Miller, and Chethan Pandarinath. Stabilizing brain-computer interfaces through alignment of latent dynamics. bioRxiv, pages 2022-04, 2022.
  13. Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy Hospedales. Learning to generalize: Meta-learning for domain generalization. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  14. Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, et al. The Kaldi speech recognition toolkit. In IEEE 2011 workshop on automatic speech recognition and understanding, number CONF. IEEE Signal Processing Society, 2011.
  15. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  16. Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2):123- 146, 1995.
  17. David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances in Neural Information Processing Systems, 32, 2019.
  18. Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731, 2019. · arXiv
  19. Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In Computer Vision-ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part III 14, pages 443-450. Springer, 2016.
  20. Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In International conference on machine learning, pages 9229-9248. PMLR, 2020.
  21. Guy H Wilson, Francis R Willett, Elias A Stein, Foram Kamdar, Donald T Avansino, Leigh R Hochberg, Krishna V Shenoy, Shaul Druckmann, and Jaimie M Henderson. Long-term unsupervised recalibration of cursor BCIs. bioRxiv, pages 2023-02, 2023.
  22. Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10687-10698, 2020.

Cited by

Sorted by citations

Related

Authors
Fan, C., Hahn, N., Kamdar, F., Avansino, D., Wilson, G. H., Hochberg, L., Shenoy, K. V., Henderson, J. M., & Willett, F. R.
Affiliations
  • Stanford University (US)
  • Stanford Medicine (US)
  • Howard Hughes Medical Institute (US)
  • Harvard University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
DOI
10.48550/arxiv.2311.03611
arXiv
2311.03611
PMC
PMC10659441
OpenAlex
W4388555240
Cited by
19
References
27
Citations by year
2026: 1 · 2025: 9 · 2024: 9
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • PubMed Central (submittedVersion)
  • arXiv (Cornell University)
Funders
  • U.S. Department of Veterans Affairs (A2295R)
  • Office of Research and Development
  • National Institutes of Health (1r01eb028171-01)
  • Wu Tsai Neurosciences Institute, Stanford University
  • National Institute on Deafness and Other Communication Disorders (R01DC014034, U01- DC017844)
  • National Institute of Biomedical Imaging and Bioengineering
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
  • Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neuroscience and Neural Engineering
  • Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
Keywords
intracortical brain-computer interfaces, pseudo-labeling, large language models, amyotrophic lateral sclerosis, continual learning, plug-and-play interface

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review