The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Intent Drift in LLM-Assisted BCI Communication: An In-Silico Benchmark Under Simulated Decoder Corruption
Abstract
Background: Large language models (LLMs) are increasingly used to correct noisy, error-prone text typed through brain-computer interfaces (BCIs) and other assistive communication devices. A fluent correction can still be wrong: the model can substitute a different intended message, a failure we call intent drift. Whether meaning survives correction, and whether model confidence flags failure, is unmeasured. Methods: We built an in-silico benchmark: 20 open-weight LLMs corrected simulated P300 speller text under five levels of decoder error (0-40% character error rate). Testing used the full ALS message-banking vocabulary, a set of clinically important messages (eg, involving a medication dose), and matched controls. Each of 4,252,326 outputs was scored faithful, degraded, or drifted by an automated system benchmarked against physicians. A separate subanalysis compared six correction strategies across seven models. Findings: Drift rose steeply with decoder error, from 2.2% at no error to 60.3% at the most severe level tested (odds ratio 2.30 per 10-percentage-point increase)-a stress-test ceiling, not an expected real-world rate. Model confidence distinguished correct from incorrect outputs reasonably well (AUROC 0.83) but overstated its own reliability: 28.4% of high-confidence outputs were not faithful. Clinically important messages drifted slightly more than matched controls (odds ratio 1.10). At low error rates, corrections succeeded more often than they went confidently wrong, but this reversed above 20-30% error. No correction strategy eliminated drift: cautious approaches reduced it, permissive ones increased it, and even the best still produced a wrong message in 18 of 100 corrections. Physician review of a sample agreed moderately with automated scoring (kappa 0.41); adjusting for this disagreement lowered but did not remove the pattern of rising drift with decoder error (31.4% to 28.3%). Interpretation: LLMs correcting BCI text produced fluent but sometimes wrong messages, more often as decoding quality worsened, and their own confidence did not reliably warn when this happened. These in-silico findings support evaluating such systems for meaning preservation, not only speed and accuracy, before deployment; prospective, human-in-the-loop evaluation is needed. Funding: Harvard Catalyst (CTSA UL1TR002541). Research in context: Evidence before this study: We searched PubMed, arXiv, and bioRxiv from database inception to July 19, 2026, combining terms for large language models with brain-computer interfaces, P300 spellers, and augmentative and alternative communication, without language restriction. Exact strings and record counts are in eMethods S9. Prior studies integrating large language models with P300 spellers and communication BCIs reported keystroke savings, typing speed, information transfer rate, and character-level accuracy. Some concluded that correction was near-optimal once residual errors were manually fixed, placing meaning-level change outside their frame. No study had measured how often post-editing changed a message’s intent, whether this depended on decoder corruption, or whether stated confidence tracked whether the output was faithful. Added value of this study: This in-silico benchmark measures intent drift and confidence calibration in language-model post-editing as a function of decoder corruption, reported separately across AUTH, a message-critical set, and matched controls. Using an empirical P300 confusion matrix and 4,252,326 labeled generations across 20 open-weight models, the primary benchmark questions found detected drift rose steeply with corruption and stated confidence discriminated faithful outputs well but was poorly calibrated; secondary analyses found message-critical content carried excess drift that survived detector removal and that detected drift varied more than two-fold across models (20.8-49.3% in AUTH). A separate subanalysis of six correction strategies (original seven-model panel) found no strategy removed drift: conservative editing or abstention lowered it, while alternatives or expansion raised it. Implications of all the available evidence: Evaluations of language-model-assisted communication BCIs and AAC systems should report intent drift and confidence calibration alongside speed and accuracy, should test message-critical content separately from routine messages, and should treat interface policy as a measured design variable. Because the construct is communicative intent, the absence of patient and public involvement in probe-set design is material. Because confidence did not reliably flag drift, it alone cannot gate human review. These are in-silico findings; they support the need for prospective, human-in-the-loop evaluation before deployment implications can be drawn.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
ESTIMATING PREVALENCE FROM THE RESULTS OF A SCREENING TEST
ROGAN, W. J., & GLADEN, B. (1978) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112510 · Cited by 1,001
Tian, K., Mitchell, E., Zhou, A., Sharma, A., Rafailov, R., Yao, H., Finn, C., & Manning, C. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.330 · Cited by 123
The case for 4-bit precision: k-bit Inference Scaling Laws
Dettmers, T., & Zettlemoyer, L. (2022) · arXiv · doi:10.48550/arxiv.2212.09720 · Cited by 46
Tian, K., Mitchell, E., Zhou, A., Sharma, A., Rafailov, R., Yao, H., Finn, C., & Manning, C. D. (2023) · arXiv · doi:10.48550/arxiv.2305.14975 · Cited by 5
Research works printed but not linked: 25
- 10.7554/eLife.18554 · DOI · Paper page
- 10.1056/NEJMoa1608085 · DOI · Paper page
- 10.1016/0013-4694(88)90149-6 · DOI · Paper page
- 10.1088/2057-1976/ae737b · DOI
- 10.1101/2025.11.06.686984 · DOI
- 10.1101/2025.10.28.685216 · doi:10.1101/2025.10.28.685216
- Parthasarathy N , Soetedjo J , Panchavati S , Parthasarathy N , Arnold C , Pouratian N , et al. Evaluation of P300 speller performance using large language models along with cross-subject training. arXiv:2410.15161. Preprint posted online 2024. · arXiv
- 10.1038/s41467-024-53873-3 · DOI
- Di Paola A , Muraro S , Marinelli R , Pilato C . Foundation models in augmentative and alternative communication: opportunities and challenges. arXiv:2401.08866. Preprint posted online 2024. · arXiv
- 10.1145/3544548.3581560 · DOI · Paper page
- 10.1038/s41591-024-03425-5 · DOI · Paper page
- 10.1038/s41591-020-1034-x · DOI · Paper page
- 10.1080/07434618.2021.2021554 · DOI
- 10.13026/0byy-ry86 · doi:10.13026/0byy-ry86
- 10.1109/TBME.2004.827072 · DOI · Paper page
- 10.18653/v1/2020.acl-main.173 · DOI · Paper page
- Reimers N , Gurevych I. Sentence-BERT: sentence embeddings using Siamese BERT-networks. arXiv:1908.10084. Preprint posted online 2019. · DOI · Paper page
- Guo C , Pleiss G , Sun Y , Weinberger KQ . On calibration of modern neural networks. arXiv:1706.04599. Preprint posted online 2017. · arXiv
- Kadavath S , Conerly T , Askell A , Henighan T , Drain D , Perez E , et al. Language models (mostly) know what they know. arXiv:2207.05221. Preprint posted online 2022. · arXiv
- 10.1016/0197-2456(86)90046-2 · DOI · Paper page
- 10.3390/s25133987 · DOI
- Jiang X , Zhou C , Duan Y , Zhao Z , Do T , Lin CT. Neural spelling: a spell-based BCI system for language neural decoding. arXiv:2501.17489. Preprint posted online 2025. · arXiv · doi:10.1109/tbme.2026.3691322
- Jo H , Yang Y , Han J , Duan Y , Xiong H , Lee WH. Are EEG-to-text models working? arXiv:2405.06459. Preprint posted online 2024. · arXiv · doi:10.2139/ssrn.4882932
- Wang Y , Wang H , Guo Y , Yang H , Li X. Escaping the BLEU trap: a signal-grounded framework with decoupled semantic guidance for EEG-to-text decoding. arXiv:2603.03312. Preprint posted online 2026. · arXiv
- 10.1145/3571730 · DOI · Paper page
Metadata
- Authors
- Gorenshtein, A., Omar, M., Jia, E., Adiniaev, Y., Daniel, O., Kruskal, J., Ahmed, M., Brook, O., Klang, E., & Barash, Y.
- Publisher
- openRxiv
- Published
- 2026-08-27
- Cited by
- 0
- References
- 5
- Type
- Preprint
- Funders
- National Center for Advancing Translational Sciences (UL1TR002541)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Intent Drift in LLM-Assisted BCI Communication: An In-Silico Benchmark Under Simulated Decoder Corruption. https://app.cochat.ai/papers/10.64898/2026.08.20.26360939
- The paper
- Gorenshtein, A., Omar, M., Jia, E., Adiniaev, Y., Daniel, O., Kruskal, J., Ahmed, M., Brook, O., Klang, E., & Barash, Y. (2026). Intent Drift in LLM-Assisted BCI Communication: An In-Silico Benchmark Under Simulated Decoder Corruption. https://doi.org/10.64898/2026.08.20.26360939
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.