The Cybathlon BCI race: Successful longitudinal mutual learning with two tetraplegic users

May combine two claims.

These learning effects were achieved under uncontrolled circumstances at the pilot's homes with minimal expert personnel intervention, while the learned outcome was replicated at a demanding international competition under adverse circumstances.

“Importantly, these learning effects were achieved under uncontrolled circumstances at the pilot’s homes with minimal expert personnel intervention, while the learned outcome was replicated at a demanding international competition—the first of its kind—under adverse circumstances, where our pilots were able to excel.”

May combine two claims.

P1 experienced a momentary loss of BCI control and had to compromise with the fourth place in the final (189.8 s).

“P1 experienced a momentary loss of BCI control and had to compromise with the fourth place in the final (189.8 s).”

May combine two claims.

Command accuracy increased for both pilots and all command types, as shown by significant positive correlations of overall accuracy to the chronological race index (P1: r = 0.70, p < .001, N = 162 races; P2: r = 0.66, p < .001, N = 45 races).

“Fig 3A verifies increasing trends of command accuracy for both pilots and all command types. This can be quantified by significant positive correlations of the overall accuracy to the (chronological) race index (P1: r = 0.70, p < .001, N = 162 races; P2: r = 0.66, p < .001, N = 45 races).”

May combine two claims.

P1 achieved slightly better average and record performances than P2.

“P1 achieved slightly better average and record performances, while P2 exhibited superior stability, having race time standard deviation of 12.9 s in the last 5 sessions (N = 28), as opposed to 20.6 s for P1 (N = 50).”

May combine two claims.

P1 qualified with 90.1 s, a performance that set the competition record, almost 32 s ahead of the second-best time belonging to P2 (122.5 s).

“P1 qualified with 90.1 s, a performance that set the competition record, almost 32 s ahead of the second-best time belonging to our second pilot, P2 (122.5 s).”

May combine two claims.

The overall discriminancy of the pilots' SMRs correlates well with total command accuracy (P1: r = 0.56, p < .001, N = 162; P2: r = 0.37, p = .013, N = 45), average pad crossing time (P1: r = −0.42, p < .001, N = 162; P2: r = −0.45, p = .002, N = 45), and race completion time (P1: r = −0.39, p < .001, N = 162; P2: r = −0.29, p = .0056, N = 45).

“The overall discriminancy of our pilots’ SMRs (average of medial and lateral locations for P1, lateral for P2) correlates well with the total command accuracy (P1: r = 0.56, p < .001, N = 162; P2: r = 0.37, p = .013, N = 45), the average pad crossing time (P1: r = −0.42, p < .001, N = 162; P2: r = −0.45, p = .002, N = 45), and the race completion time (P1: r = −0.39, p < .001, N = 162; P2: r = −0.”

May combine two claims.

Feature discriminancy significantly increased only in control paradigm 4 (0.27 ± 0.07 to 0.34 ± 0.05, p = .045, two-sided Wilcoxon ranksum test), while no difference or even a reduction was reported for the other two paradigms.

“Interestingly, the discriminancy significantly increased only in the case of control paradigm 4 (0.27 ± 0.07 to 0.34 ± 0.05, p = .045, two-sided Wilcoxon ranksum test), while no difference (or even a reduction) is reported for the other two paradigms.”

May combine two claims.

The median pad crossing time performances for P1/P2 across all races were 4.9/4.4 s (N = 853/205) for spin, 4.1/4.9 s (N = 766/198) for jump, and 6.2/7.2 s (N = 463/196) for slide.

“The illustrated median pad crossing time performances (for P1/P2) across all races (training and competition) were 4.9/4.4 s (N = 853/205) for spin, 4.1/4.9 s (N = 766/198) for jump, and 6.2/7.2 s (N = 463/196) for slide”

May combine two claims.

Both pilots exhibited significant command accuracy increase in all individual tasks, with the only exception being the spin command for P2, which had stable accuracy.

“Both pilots exhibited significant command accuracy increase in all individual tasks (the only exception being the spin command for P2, with stable accuracy).”

May combine two claims.

None of the six responding competing teams invested considerably in facilitation of subject learning and they were mainly concentrated on the machine learning side.

“In support of our hypothesis, it seems that none of these teams invested considerably in the facilitation of subject learning and were mainly concentrated on the machine learning side.”

May combine two claims.

Both these improvements (P1 and P2 race completion time reductions) are statistically significant (p < .001, two-sided Wilcoxon ranksum tests).

“Both these improvements are statistically significant (p < .001, two-sided Wilcoxon ranksum tests).”

May combine two claims.

The incremental mutual learning procedure has been very effective in bringing up an emerging SMR pattern (high β-band, 22-32 Hz) for both pilots, coherent with both hands MI and both feet MI locations of the sensorimotor cortex.

“Fig 4A demonstrates that our incremental mutual learning procedure has been very effective in bringing up an emerging SMR pattern (high β-band, 22–32 Hz) for both pilots, coherent with both hands MI (lateral, electrodes FC3, C3, CP3, FC4, C4, CP4 of the 10–20 EEG system) and both feet MI (medial, electrodes FCz, Cz, CPz) locations of the sensorimotor cortex”

May combine two claims.

The race completion times of P1 and P2 throughout training averaged 126.9 ± 21.3 (N = 182) s and 130.3 ± 22.9 (N = 57) s, with all-time records of 83.3 and 86.3 s, respectively.

“The race completion times of our pilots throughout training (Fig 1C) averaged 126.9 ± 21.3 (N = 182) s for P1 and 130.3 ± 22.9 (N = 57) s for P2, with all-time records of 83.3 and 86.3 s, respectively.”

May combine two claims.

Pilot P2 largely maintained the same brain pattern in both competition races, even increasing the strength of medial modulation in the final (channels Cz and CPz).

“On the contrary, pilot P2 largely maintained the same brain pattern in both competition races, even increasing the strength of medial modulation in the final (channels Cz and CPz, both channels were selected for the classifier used in the competition).”

May combine two claims.

P2 exhibited superior stability, having race time standard deviation of 12.9 s in the last 5 sessions (N = 28), as opposed to 20.6 s for P1 (N = 50).

“P1 achieved slightly better average and record performances, while P2 exhibited superior stability, having race time standard deviation of 12.9 s in the last 5 sessions (N = 28), as opposed to 20.6 s for P1 (N = 50).”

May combine two claims.

Early attempts with a 3-class BCI (paradigm 1) severely compromised the total command accuracy, which is reflected in the high race completion times during this period.

“As shown in Fig 1C, early attempts with a 3-class BCI (paradigm 1) severely compromised the total command accuracy (Fig 3A), which is reflected in the high race completion times during this period.”

May combine two claims.

The median crossing time of white pads was 10.7 s for P1 (N = 510) and 8.4 s for P2 (N = 151).

“The median crossing time of white pads was 10.7 s and 8.4 s for P1 (N = 510) and P2 (N = 151), respectively—far below the worst-case scenario of 19 s and closer to the optimum of 5.5 s.”

May combine two claims.

There is a significant enhancement trend of these patterns' discriminancy over runs for P1 (N = 214: r = 0.47, p < .001 for medial and r = 0.44, p < .001 for lateral locations) and P2 (N = 79: r = 0.47, p < .001 for medial and r = 0.64, p < .001 for lateral locations), accounting for considerable, statistically significant increase for both pilots and locations between the first and last four runs

“Fig 4B further substantiates a significant enhancement trend of these patterns' discriminancy over runs (P1, N = 214: r = 0.47, p < .001 for medial and r = 0.44, p < .001 for lateral locations; P2, N = 79: r = 0.47, p < .001 for medial and r = 0.64, p < .001 for lateral locations), accounting for considerable, statistically significant increase for both pilots and locations between the first and”

Probably not a substantial claim: may not pass S3 (load-bearing).

The closest times belonging to the pilots of other competing teams throughout the tournament were 132, 135, 136, and 146 s.

“The closest times belonging to the pilots of other competing teams throughout the tournament were 132, 135, 136, and 146 s.”

May combine two claims.

Our results showcase strong and continuous learning effects at all targeted levels-machine, subject, and application-with both end-users over a longitudinal study lasting several months.

“Our results showcase strong and continuous learning effects at all targeted levels—machine, subject, and application—with both end-users over a longitudinal study lasting several months.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Pilot P2 also reported reaching automatic BCI control.

“This was also reported by our pilot P2 [44].”

• cites [44] S Perdikis, 2017, Brain racers, IEEE Spectrum sourceNot checked yet

  Not checked in the free run.

May combine two claims.

During races, BCI commands always coincide with the presence of SMR, which has to be sufficiently large for the BCI to reach the decision threshold.

“Indeed, during races, BCI commands always coincide with the presence of SMR, which has to be sufficiently large for the BCI to reach the decision threshold.”

May combine two claims.

P1 exhibited a significant positive trend only in the case of the last control paradigm; in the other cases, discriminancy remains stable or even decreases over time.

“In fact, the user not only exhibited a general improvement of the features’ separability from the initial design to the final one (from control paradigm 1 to 4, Fig 6D) but also a significant positive trend only in the case of the last control paradigm. In the other cases, discriminancy remains stable (or even decreases) over time.”

May combine two claims.

Although the BCI did not deliver a command every time the pilot generated an SMR, whenever the BCI delivered it, the pilot was eliciting an SMR.

“Thus, although the BCI did not deliver a command to the avatar every time the pilot generated an SMR, the opposite holds: whenever the BCI delivered it, the pilot was eliciting an SMR.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Both pilots use other advanced assistive technology in their daily lives, such as driving aids and speech-to-text software.

“They both use other advanced AT in their daily lives, like driving aids and speech-to-text software.”

May combine two claims.

The SMR brain patterns of both participants for the different training stages resemble each other but are enhanced in magnitude of discriminancy, suggesting that they both gradually adapted to the increasing task demands.

“S2 Fig illustrates how the SMR brain patterns of both participants for these different stages resemble each other but are enhanced in magnitude of discriminancy, suggesting that they both gradually adapted to the increasing task demands.”

May combine two claims.

Although P1 showcased better average performance, he also exhibited higher variability than P2.

“As shown, although P1 showcased better average performance, he also exhibited higher variability than P2.”

May combine two claims.

Both pilots were found to optimally modulate both hands and both feet MI, so offline runs were limited to these tasks to collect clean data for updating the BCI algorithm's parameters.

“Subsequently, offline runs were limited to both hands and both feet MI, which both our pilots were found to optimally modulate, so as to collect “clean” data for updating the BCI algorithm’s parameters.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

P1 unsuccessfully tried imagination of tongue movement, as well as "word" and "mathematical association" mental tasks.

“P1 has also unsuccessfully tried imagination of tongue movement, as well as “word” and “mathematical association” mental tasks.”

May combine two claims.

The competitors' approach involved frequent classifier recalibration and feature re-selection, as well as training protocols that were relatively short and/or not particularly intense.

“Indeed, their approach involved frequent classifier recalibration and feature re-selection, as well as training protocols that were relatively short and/or not particularly intense.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Both participants are able to stabilize their neck and head, but only P2 can also stabilize his trunk.

“They are both able to stabilize their neck and head, but only P2 can also stabilize his trunk.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Both end-users were under medication for the treatment of spasms and other symptoms related to their medical condition.

“Both end-users were under medication for the treatment of spasms and other symptoms related to their medical condition.”

May combine two claims.

Both participants maintain no control of the lower limbs and only limited control of the upper limbs.

“Both participants maintain no control of the lower limbs and only limited control of the upper limbs.”

May combine two claims.

Both pilots sustained complete lesions at level C5-C6 and scored "A" on the ASIA impairment scale.

“Both our pilots—48-year-old P1, injured in December 1989, and 30-year-old P2, injured in May 2003—have sustained complete lesions at level C5–C6 and have scored “A” (Complete injury—No motor or sensory function is preserved in the sacral segments S4 or S5) in the ASIA impairment scale.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Neither pilot carries pacemakers or other implants, suffers epilepsy or cyber-sickness, or needs respiratory assistance.

“Neither of our pilots carries pacemakers or other implants, suffers epilepsy or cyber-sickness, or needs respiratory assistance.”

May combine two claims.

Residual motor abilities included, for both pilots, unaffected bilateral control of shoulder and elbow movements and compromised control of wrist movements, while neither maintained control over the fingers.

“Residual motor abilities included, for both pilots, unaffected bilateral control of shoulder and elbow movements and compromised control of wrist movements, while neither of the two maintained control over the fingers.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

P1 had several years prior participated in MI BCI studies, while P2 was BCI naive at the onset of his Cybathlon training.

“P1 had several years prior participated in the MI BCI studies reported in [35,51,71], while P2 was BCI naive at the onset of his Cybathlon training.”

• cites [35] R Leeb, 2015, Towards Independence: A BCI Telepresence Robot for People With Severe Motor Disabilities, Proc IEEE sourceNot checked yet

  Not checked in the free run.

• cites [51] R Leeb, 2013, Transferring brain-computer interfaces beyond the laboratory: Successful application control for motor-disabled users, Artif Intell Med sourceNot checked yet

  Not checked in the free run.

• cites [71] S Perdikis, 2014, Clinical evaluation of BrainTree, a motor imagery hybrid BCI speller, J Neural Eng sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The study showcased continuous and consolidated learning on the machine side and on the user side, as well as an effect of application training on subject learning.

“In conclusion, the Cybathlon 2016 provided the ideal framework to implement and evaluate the effects of longitudinal mutual learning, which allowed us to showcase continuous and consolidated learning not only on the machine side (which is regularly well documented) but also—and most importantly—on the user side, as well as an effect of application training on subject learning.”

May combine two claims.

The unidentified high-frequency component concerns two features selected for P2's BCI control (CP3/30 Hz and CP3/32 Hz) and approximately one-fourth of the executed runs.

“This effect only concerns two of the features selected for P2’s BCI control (CP3/30 Hz and CP3/32 Hz) and approximately one-fourth of the executed runs.”

May combine two claims.

All learning indices (subject's and machine's), as well as application performances, correlate with the amount of training and with one another, which establishes that individual subject and machine learning improvements drive the enhancement of BCI-actuated application control.

“Importantly, all learning indices (the subject’s and the machine’s), as well as application performances, can be shown to correlate with the amount of training and with one another, which establishes that the individual subject and machine learning improvements are not irrelevant but actually drive the enhancement of BCI-actuated application control.”

Not yet classified.

The 3-class and 2-class BCI control paradigms (paradigms 1 and 2) had unsatisfactory outcomes.

“Given the unsatisfactory outcome of these two approaches, another two paradigms were designed, both investigating well-known human–computer interaction principles for supporting all three avatar commands given only a binary input.”

May combine two claims.

Direct evidence of learned SMR modulation at the BCI feature level is rare or incomplete, derived in able-bodied populations and not longitudinal.

“Direct evidence of learned SMR modulation at the BCI feature level is, in fact, rare or incomplete, derived in able-bodied populations and not longitudinal [10,23,26,28,38–41].”

• cites [10] C Neuper, 2003, Clinical application of an EEG-based brain–computer interface: a case study in a patient with severe motor impairment, Clin Neurophysiol sourceNot checked yet

  Not checked in the free run.

• cites [23] JD Wander, 2013, Distributed cortical adaptation during learning of a brain-computer interface task, Proc Natl Acad Sci U S A sourceNot checked yet

  Not checked in the free run.

• cites [26] C Vidaurre, 2011, Machine-learning-based coadaptive calibration for brain-computer interfaces, Neural Comput sourceNot checked yet

  Not checked in the free run.

• cites [28] C Vidaurre, 2011, Co-adaptive calibration to improve BCI efficiency, J Neural Eng sourceNot checked yet

  Not checked in the free run.

• cites [38] J Faller, 2012, Autocalibration and recurrent adaptation: Towards a plug and play online ERD-BCI, IEEE Trans Neural Syst Rehabil Eng sourceNot checked yet

  Not checked in the free run.

• cites [39] J Faller, 2014, A co-adaptive brain-computer interface for end users with severe motor impairment, PLoS ONE sourceNot checked yet

  Not checked in the free run.

• cites [40] JR Wolpaw, 2004, Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans, Proc Natl Acad Sci U S A sourceNot checked yet

  Not checked in the free run.

• cites [41] DJ McFarland, 2008, Emulation of computer mouse control with a noninvasive brain–computer interface, J Neural Eng sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The emergence of interfaces based on evoked responses (P300, SSVEP) as the most efficient BCI solution for communication has promoted the use of machine learning because the margin for humans to learn to regulate evoked potentials is considered narrow.

“On the one hand, the emergence of interfaces based on evoked responses (P300, steady-state visually evoked potentials [SSVEP]) as the most efficient BCI solution for communication [13–16] has promoted the use of machine learning because the margin for humans to learn to regulate evoked potentials is considered to be narrow.”

• cites [13] EW Sellers, 2010, A brain-computer interface for long-term independent home use, Amyotroph Lateral Scler sourceNot checked yet

  Not checked in the free run.

• cites [14] EW Sellers, 2014, Noninvasive brain-computer interface enables communication after brainstem stroke, Sci Transl Med sourceNot checked yet

  Not checked in the free run.

• cites [15] EW Sellers, 2006, P300-based brain-computer interface: initial tests by ALS patients, ANot checked yet

  Not checked in the free run.

• cites [16] X Chen, 2015, High-speed spelling with a noninvasive brain-computer interface, Proc Natl Acad Sci U S A sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Successful BCI requires that the user and the embedded decoder engage in a mutual learning process, in which users must learn to generate distinct brain patterns for different mental tasks, while machine learning techniques ought to discover, interpret, and allow a model's adaptation to the potentially changing individual brain patterns associated to these tasks.

“Under this view, successful BCI requires that the user and the embedded decoder engage in a mutual learning process, in which users must learn to generate distinct brain patterns for different mental tasks, while machine learning techniques ought to discover, interpret, and allow a model’s adaptation to the potentially changing individual brain patterns associated to these tasks [22].”

• cites [22] JdR Millán, 2015, Brain-Machine Interfaces: The Perception-Action Closed Loop: A Two-Learner System, IEEE Systems, Man, and Cybernetics Magazine sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Co-adaptive interfaces have been proposed as a remedy and increasingly adopted and modeled as a training strategy.

“On the contrary, co-adaptive (a term we use interchangeably to mutual learning) interfaces, in which the capacities of both learning agents—the brain and the machine—are accommodated and coordinated, has been very early proposed as a remedy [18] and more recently increasingly adopted and modeled as a training strategy [19–21].”

• cites [18] JR Wolpaw, 2002, Brain-computer interfaces for communication and control, Clin Neurophysiol sourceNot checked yet

  Not checked in the free run.

• cites [19] SV Hiremath, 2015, Brain computer interface learning for systems based on electrocorticography and intracortical microelectrode arrays, Front Integr Neurosci sourceNot checked yet

  Not checked in the free run.

• cites [20] JS Merel, 2013, Adv Neural Inf Process SystNot checked yet

  Not checked in the free run.

• cites [21] J Mladenović, 2018, Brain-Computer Interfaces Handbook: Technological and Theoretical AdvancesNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

More than half of published BCI works research signal-processing and machine-learning methods.

“This is substantiated by the fact that more than half of published BCI works research signal-processing and machine-learning methods [12].”

• cites [12] H-J Hwang, 2013, EEG-Based Brain-Computer Interfaces: A Thorough Literature Survey, Int J Hum Comput Interact sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Increased SMR modulation (discriminancy) seems to be crucial for enhanced BCI and, through the latter, also application performances.

“Hence, increased SMR modulation (discriminancy) seems to be crucial for enhanced BCI and, through the latter, also application performances.”

May combine two claims.

During the races comparing paradigms 3 and 4, P1 alternated between paradigms 3 and 4, and for this reason, we cannot expect strong subject learning effects.

“Importantly, during these races, P1 alternated between paradigms 3 and 4 (Fig 1A), and for this reason, we cannot expect strong subject learning effects.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Users have reported reaching a state of proficiency through learning where BCI control becomes "automatic," as they no longer need to engage explicitly in MI.

“Interestingly, users have reported reaching a state of proficiency through learning where BCI control becomes “automatic,” as they no longer need to engage explicitly in MI [23,40–42,50,60,61].”

• cites [23] JD Wander, 2013, Distributed cortical adaptation during learning of a brain-computer interface task, Proc Natl Acad Sci U S A sourceNot checked yet

  Not checked in the free run.

• cites [40] JR Wolpaw, 2004, Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans, Proc Natl Acad Sci U S A sourceNot checked yet

  Not checked in the free run.

• cites [41] DJ McFarland, 2008, Emulation of computer mouse control with a noninvasive brain–computer interface, J Neural Eng sourceNot checked yet

  Not checked in the free run.

• cites [42] DJ McFarland, 2010, Electroencephalographic (EEG) control of three-dimensional movement, J Neural Eng sourceNot checked yet

  Not checked in the free run.

• cites [50] M Tavella, 2010, Millán JdR. Towards natural noninvasive hand neuroprostheses for daily living, Conf Proc IEEE Eng Med Biol SocNot checked yet

  Not checked in the free run.

• cites [60] E Curran, 2003, Learning to control brain activity: A review of the production and control of EEG components for driving brain–computer interface (BCI) systems, Brain Cogn sourceNot checked yet

  Not checked in the free run.

• cites [61] LA Miner, 1998, Answering questions with an electroencephalogram-based brain-computer interface, Arch Phys Med Rehabil sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Evidence from invasive and semi-invasive BCI shows that learning and co-adaptation have been well documented.

“Another similar, overstated extrapolation regards evidence from invasive and semi-invasive BCI, where learning and co-adaptation have been well documented [19,23,24,57–59].”

• cites [19] SV Hiremath, 2015, Brain computer interface learning for systems based on electrocorticography and intracortical microelectrode arrays, Front Integr Neurosci sourceNot checked yet

  Not checked in the free run.

• cites [23] JD Wander, 2013, Distributed cortical adaptation during learning of a brain-computer interface task, Proc Natl Acad Sci U S A sourceNot checked yet

  Not checked in the free run.

• cites [24] B Jarosiewicz, 2008, Functional network reorganization during learning in a brain-computer interface paradigm, Proc Natl Acad Sci U S A sourceNot checked yet

  Not checked in the free run.

• cites [57] K Ganguly, 2009, Emergence of a stable cortical map for neuroprosthetic control, PLoS Biol sourceNot checked yet

  Not checked in the free run.

• cites [58] JM Carmena, 2003, Learning to control a brain-machine interface for reaching and grasping by primates, PLoS Biol sourceNot checked yet

  Not checked in the free run.

• cites [59] AL Orsborn, 2014, Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control, Neuron sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Subject learning in online MI BCI is most often hypothesized to occur "by default" in analogy to neurofeedback training.

“To begin with, subject learning in online MI BCI is most often hypothesized to occur “by default” in analogy to neurofeedback training [1,46,54,55].”

• cites [1] N Birbaumer, 1999, A spelling device for the paralysed, Nature sourceNot checked yet

  Not checked in the free run.

• cites [46] C Neuper, 2009, Neurofeedback Training for BCI Control, The Frontiers Collection sourceNot checked yet

  Not checked in the free run.

• cites [54] G Pfurtscheller, 2003, “Thought”–control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia, Neurosci Lett sourceNot checked yet

  Not checked in the free run.

• cites [55] N Birbaumer, 2006, Breaking the silence: Brain-computer interfaces (BCI) for communication and motor control, Psychophysiology sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

Claims of automatic BCI control are qualitative and do not constitute hard evidence for the existence of subject learning.

“However, such claims are rather qualitative and do not constitute hard evidence for the existence of subject learning.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Some works find no evident learning effects at the neural correlate level.

“Some works find no evident learning effects at the neural correlate level [25,34].”

• cites [25] C Vidaurre, 2010, Towards a cure for BCI illiteracy, Brain Topogr sourceNot checked yet

  Not checked in the free run.

• cites [34] J Höhne, 2014, Motor imagery for severely motor-impaired patients: evidence for brain-computer interfacing as superior control solution, PLoS ONE sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

Subject learning must be accompanied by increasing and consolidated separability of brain patterns in order to drive BCI performance upwards.

“We argue that this effect must still be accompanied by increasing and consolidated separability of the brain patterns in order to drive BCI performance upwards.”

May combine two claims.

No apparent "breakthroughs" are evident, which could support the only likely alternative hypothesis, that of the employment of better mental strategies sparking immediate, rather than gradual, improvements.

“No apparent “breakthroughs” are evident, which could support the only likely alternative hypothesis, that of the employment of better mental strategies sparking immediate, rather than gradual, improvements [9,64].”

• cites [9] G Pfurtscheller, 2000, Brain oscillations control hand orthosis in a tetraplegic, Neurosci Lett sourceNot checked yet

  Not checked in the free run.

• cites [64] SE Kober, 2013, Learning to modulate one’s own brain activity: The effect of spontaneous mental strategies, Front Hum Neurosci sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors' previous work reported a short-term decrease in feature discriminancy during adaptive spelling.

“Our previous work has even reported a short-term decrease in feature discriminancy during adaptive spelling [29].”

• cites [29] S Perdikis, 2016, Millán JdR. Context-aware adaptive spelling in motor imagery BCI, J Neural Eng sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Other studies have reported emergence of SMR modulations, but given the short number of experimental sessions, the observed neurophysiological patterns might only be indicative of transitory effects rather than consolidated subject learning.

“Other studies have reported emergence of such SMR modulations [26,28,38,39], but given the short number of experimental sessions they carried out, the observed neurophysiological patterns might only be indicative of transitory effects rather than consolidated subject learning.”

• cites [26] C Vidaurre, 2011, Machine-learning-based coadaptive calibration for brain-computer interfaces, Neural Comput sourceNot checked yet

  Not checked in the free run.

• cites [28] C Vidaurre, 2011, Co-adaptive calibration to improve BCI efficiency, J Neural Eng sourceNot checked yet

  Not checked in the free run.

• cites [38] J Faller, 2012, Autocalibration and recurrent adaptation: Towards a plug and play online ERD-BCI, IEEE Trans Neural Syst Rehabil Eng sourceNot checked yet

  Not checked in the free run.

• cites [39] J Faller, 2014, A co-adaptive brain-computer interface for end users with severe motor impairment, PLoS ONE sourceNot checked yet

  Not checked in the free run.

May combine two claims.

These works report on longitudinal training and involve end-users.

“These works report on longitudinal training and involve end-users.”

• cites [9] G Pfurtscheller, 2000, Brain oscillations control hand orthosis in a tetraplegic, Neurosci Lett sourceNot checked yet

  Not checked in the free run.

• cites [10] C Neuper, 2003, Clinical application of an EEG-based brain–computer interface: a case study in a patient with severe motor impairment, Clin Neurophysiol sourceNot checked yet

  Not checked in the free run.

• cites [42] DJ McFarland, 2010, Electroencephalographic (EEG) control of three-dimensional movement, J Neural Eng sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

Deriving an index of neuroimaging-based separability at the feature level to quantify the user's BCI aptitude and its evolution over time is a sine qua non prerequisite for corroborating the existence of subject learning.

“Hence, we consider that deriving some index of neuroimaging-based separability at the feature level in order to quantify the user’s BCI aptitude (and its evolution over time) is a sine qua non prerequisite for corroborating the existence of subject learning [46].”

• cites [46] C Neuper, 2009, Neurofeedback Training for BCI Control, The Frontiers Collection sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Better BCI performance could be due to decoder recalibration, re-parameterizations of the BCI, and the application and adoption of better mental strategies, among other factors.

“Better performance could be due to decoder recalibration [29], re-parameterizations of the BCI, and the application and adoption of better mental strategies [9,64,65], among other factors.”

• cites [29] S Perdikis, 2016, Millán JdR. Context-aware adaptive spelling in motor imagery BCI, J Neural Eng sourceNot checked yet

  Not checked in the free run.

• cites [9] G Pfurtscheller, 2000, Brain oscillations control hand orthosis in a tetraplegic, Neurosci Lett sourceNot checked yet

  Not checked in the free run.

• cites [64] SE Kober, 2013, Learning to modulate one’s own brain activity: The effect of spontaneous mental strategies, Front Hum Neurosci sourceNot checked yet

  Not checked in the free run.

• cites [65] S Perdikis, 2014, Subject-oriented training for motor imagery brain-computer interfaces, Conf Proc IEEE Eng Med Biol SocNot checked yet

  Not checked in the free run.

May combine two claims.

These works do not explicitly relate induced brain rhythm changes to BCI performance or show that SMR improvements were consistent and continuous.

“Nevertheless, these works do not explicitly relate induced brain rhythm changes to BCI performance or show that SMR improvements were consistent and continuous.”

Classified as probably not a substantial claim.

Differences in machine-learning methods of all participating teams were too subtle to explain the competition outcomes according to the organizers.

“Of note, the differences in machine-learning methods of all participating teams were too subtle to explain the competition outcomes according to the organizers [8].”

• cites [8] D Novak, 2018, Benchmarking Brain-Computer Interfaces Outside the Laboratory: The Cybathlon 2016, Front Neurosci sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Certified confirmation by their medical doctor of safety to participate in the Cybathlon event was requested and signed for both pilots, and insurance against accidents and injuries was taken.

“Certified confirmation by their medical doctor of safety to participate in the Cybathlon event was requested and signed for both pilots, and insurance against accidents and injuries was taken, as per Cybathlon’s regulations.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

The extracted SMR discriminancy index would be the primary descriptor of learning anyway, since learning can only be an instance of neurofeedback operant conditioning if learned brain activity modulation happens with respect to the same neural activity that is fed back to the user.

“However, it must be noted that the extracted SMR discriminancy index would be the primary descriptor of learning anyway, since the latter can only be an instance of neurofeedback operant conditioning if learned brain activity modulation happens with respect to the same neural activity that is fed back to the user (in our case, SMRs on selected channels and bands).”

Probably not a substantial claim: may not pass S3 (load-bearing).

A safety and eligibility check was conducted by the organizers the day before the competition.

“A safety and eligibility check was also conducted by the organizers the day before the competition.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Informed consents were signed in accordance with the Declaration of Helsinki, and participation in training sessions and competition was approved by the Swiss committees for ethics in human research.

“Informed consents have been signed in accordance with the Declaration of Helsinki, and their participation in the training sessions as well as in the competition has been approved by the Swiss committees for ethics in human research (protocol number PB_2017–00295, 20/15 CCVEM of the Cantonal Committee of Vaud, Switzerland for ethics in human research, CER-VD).”

May combine two claims.

The Cybathlon motivated the recruitment of two end-user participants and the involvement of a real BCI application operated in real-world circumstances, which advocates translational implications of the findings.

“Furthermore, the Cybathlon motivated the recruitment of two end-user participants and the involvement of a real BCI application operated in real-world circumstances, which advocates translational implications of our findings.”

Probably not a substantial claim: may not pass S3 (load-bearing).

This study has been approved by the Cantonal Committee of Vaud for ethics in human research under protocol number PB_2017-00295 (20/15 CCVEM).

“This study has been approved by the Cantonal Committee of Vaud (VD, Switzerland) for ethics in human research (CER-VD) under protocol number PB_2017–00295 (20/15 CCVEM).”

Probably not a substantial claim: may not pass S3 (load-bearing).

During the competition, judges controlled for violations of the prerequisite that imagination suppress any overt motor act.

“During the competition, judges controlled for violations of the latter prerequisite.”

Probably not a substantial claim: may not pass S2 (checkable).

Small but potentially crucial differences between the previous study and the present study might explain the different outcomes.

“We postulate it is mostly the small, but potentially crucial, differences between that and the present study that might explain the different outcomes.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Comparison with the competitors' known strategies is in agreement with the assessment that infrequent recalibration is preferable.

“Comparison with our competitors’ known strategies are in agreement with this assessment.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The racing track was randomized for each race, simulating the actual Cybathlon conditions.

“The racing track was randomized for each race, simulating the actual Cybathlon conditions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors' training apparatus has been very similar to the one applied in their previous work.

“We denote that our training apparatus, which certainly falls under the category of “conventional” MI BCI training protocols relying on visual feedback training on top of initial machine calibration with spontaneous SMRs, has been very similar to the one we have applied in our previous work [51].”

• cites [51] R Leeb, 2013, Transferring brain-computer interfaces beyond the laboratory: Successful application control for motor-disabled users, Artif Intell Med sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The competitors can be relied on as a fair control group because they essentially adopted a training methodology mainly based on machine learning, while the authors followed a more holistic mutual learning methodology.

“Nevertheless, we can rely on our competitors as a fair control group because they have essentially adopted a training methodology mainly based on machine learning, as per the results of the questionnaire (S1 Table), while we followed a more holistic mutual learning methodology.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

All EEG and race time data collected have been included in the statistical analysis, and no outliers have been defined.

“All EEG and race time data collected have been included into our statistical analysis, and no outliers have been defined.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Various psychological factors, such as motivation, attention, and stress, have been implicated in these negative effects, which are quite frequent in MI BCI operation.

“Various psychological factors (such as motivation, attention, and stress) have been implicated in these negative effects [2,47,53,66–68], which unfortunately are quite frequent in MI BCI operation.”

• cites [2] U Chaudhary, 2016, Brain-computer interfaces for communication and rehabilitation, Nat Rev Neurol sourceNot checked yet

  Not checked in the free run.

• cites [47] R Chavarriaga, 2017, Heading for new shores! Overcoming pitfalls in BCI design, Brain-Computer Interfaces sourceNot checked yet

  Not checked in the free run.

• cites [53] F Lotte, 2013, Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: Lessons learned from instructional design, Front Hum Neurosci sourceNot checked yet

  Not checked in the free run.

• cites [66] C Jeunet, 2016, Why standard brain-computer interface (BCI) training protocols should be changed: an experimental study, J Neural Eng sourceNot checked yet

  Not checked in the free run.

• cites [67] E Baykara, 2016, Effects of training and motivation on auditory P300 brain-computer interface performance, Clin Neurophysiol sourceNot checked yet

  Not checked in the free run.

• cites [68] EM Hammer, 2012, Psychological predictors of SMR-BCI performance, Biol Psychol sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The impact of training with the BCI application might be related to a need for getting accustomed to the actual application demands but probably also to increased user motivation provided by the gaming application and the goal of participating in an international competition.

“This might be related to a need for getting accustomed to the actual application demands [47] but probably also to increased user motivation [68] provided both by the gaming application and the goal of participating in an international competition [8].”

• cites [47] R Chavarriaga, 2017, Heading for new shores! Overcoming pitfalls in BCI design, Brain-Computer Interfaces sourceNot checked yet

  Not checked in the free run.

• cites [68] EM Hammer, 2012, Psychological predictors of SMR-BCI performance, Biol Psychol sourceNot checked yet

  Not checked in the free run.

• cites [8] D Novak, 2018, Benchmarking Brain-Computer Interfaces Outside the Laboratory: The Cybathlon 2016, Front Neurosci sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Infrequent recalibration of the BCI has been beneficial to the subject learning side while still adequately accommodating the machine learning side of the mutual learning scheme.

“Last but not least, we postulate that, despite current opinion considering this potentially detrimental to BCI accuracy, infrequent recalibration of the BCI has also been beneficial to the subject learning side (Fig 4) while still adequately accommodating the machine learning side of our mutual learning scheme (Fig 2 and Fig 3).”

• AssumptionThe authors state this as an assumption: The machine learning model was sufficiently optimized.

Probably not a substantial claim: may not pass S2 (checkable).

The Cybathlon 2016 provided the ideal framework to implement and evaluate the effects of longitudinal mutual learning.

“In conclusion, the Cybathlon 2016 provided the ideal framework to implement and evaluate the effects of longitudinal mutual learning, which allowed us to showcase continuous and consolidated learning not only on the machine side (which is regularly well documented) but also—and most importantly—on the user side, as well as an effect of application training on subject learning.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Lack of robustness is a well-known issue of all BCI paradigms and has been associated with the nonstationarity of brain signals.

“Lack of robustness is a well-known issue of all BCI paradigms and has been associated to the nonstationarity of brain signals [18,25,27].”

• cites [18] JR Wolpaw, 2002, Brain-computer interfaces for communication and control, Clin Neurophysiol sourceNot checked yet

  Not checked in the free run.

• cites [25] C Vidaurre, 2010, Towards a cure for BCI illiteracy, Brain Topogr sourceNot checked yet

  Not checked in the free run.

• cites [27] C Vidaurre, 2006, A fully on-line adaptive BCI, IEEE Trans Biomed Eng sourceNot checked yet

  Not checked in the free run.