CoChat

The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.

Bimodal electroencephalography-functional magnetic resonance imaging dataset for inner-speech recognition

Open access

Abstract

The recognition of inner speech, which could give a ‘voice’ to patients that have no ability to speak or move, is a challenge for brain-computer interfaces (BCIs). A shortcoming of the available datasets is that they do not combine modalities to increase the performance of inner speech recognition. Multimodal datasets of brain data enable the fusion of neuroimaging modalities with complimentary properties, such as the high spatial resolution of functional magnetic resonance imaging (fMRI) and the temporal resolution of electroencephalography (EEG), and therefore are promising for decoding inner speech. This paper presents the first publicly available bimodal dataset containing EEG and fMRI data acquired nonsimultaneously during inner-speech production. Data were obtained from four healthy, right-handed participants during an inner-speech task with words in either a social or numerical category. Each of the 8-word stimuli were assessed with 40 trials, resulting in 320 trials in each modality for each participant. The aim of this work is to provide a publicly available bimodal dataset on inner speech, contributing towards speech prostheses.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41597-023-02286-w

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 EEG /fMRI schedule - The overall experimental protocol performed over a period of 3 consecutive days. Day-01 contains the EEG recordings as well as the recordings of the first fMRI session of sub-01, sub-02, and sub-03. Day-02 contains the fMRI recordings for session 2 of sub-01, sub-02, and sub-03, the fMRI recordings for session 1 of sub-05 and the eeg recordings of sub-05. Day-03 contains the fMRI recordings for session 2 of sub-05. There was always a relaxation period of at least one hour in between the recordings and proper time for a break to avoid participants’ fatigue.
Fig. 1 EEG /fMRI schedule - The overall experimental protocol performed over a period of 3 consecutive days. Day-01 contains the EEG recordings as well as the recordings of the first fMRI session of sub-01, sub-02, and sub-03. Day-02 contains the fMRI recordings for session 2 of sub-01, sub-02, and sub-03, the fMRI recordings for session 1 of sub-05 and the eeg recordings of sub-05. Day-03 contains the fMRI recordings for session 2 of sub-05. There was always a relaxation period of at least one hour in between the recordings and proper time for a break to avoid participants’ fatigue.CC BY · PubMed Central
Fig. 2 Experimental paradigm - The trial designs are depicted in the top part of the figure and the respected timelines for the trials are shown at the bottom. During the fMRI inner-speech task, participants were requested to think of the presented word as many times as possible. There were 320 trials in total, split into two sessions of 160 trials each. For the eeg recordings, there was only one session, and the participants were instructed to use their inner-speech only once during the inner-speech task interval. Note that the rest period for the fMRI protocol was longer than that for the eeg protocol.
Fig. 2 Experimental paradigm - The trial designs are depicted in the top part of the figure and the respected timelines for the trials are shown at the bottom. During the fMRI inner-speech task, participants were requested to think of the presented word as many times as possible. There were 320 trials in total, split into two sessions of 160 trials each. For the eeg recordings, there was only one session, and the participants were instructed to use their inner-speech only once during the inner-speech task interval. Note that the rest period for the fMRI protocol was longer than that for the eeg protocol.CC BY · PubMed Central
Fig. 3 Components decomposed by ICA, showing eye blinks and brain activity for subject 5, before artifacts removal. Top left: topography map illustrating the projection of the independent component activity. Center: event-related potentials of the independent component and its associated power, per epoch. The times marked with vertical lines indicate the start of the fixation, inner-speech task, and rest periods (at −1,000 ms, 0 ms and 2,000 ms respectively). Bottom: spectrum of the independent component.
Fig. 3 Components decomposed by ICA, showing eye blinks and brain activity for subject 5, before artifacts removal. Top left: topography map illustrating the projection of the independent component activity. Center: event-related potentials of the independent component and its associated power, per epoch. The times marked with vertical lines indicate the start of the fixation, inner-speech task, and rest periods (at −1,000 ms, 0 ms and 2,000 ms respectively). Bottom: spectrum of the independent component.CC BY · PubMed Central
Fig. 4 Illustration of eye artifacts reduction in the frontal region. EEG signal before (blue) and after (red) applying ICA.
Fig. 4 Illustration of eye artifacts reduction in the frontal region. EEG signal before (blue) and after (red) applying ICA.CC BY · PubMed Central
Fig. 5 Plots of event-related potentials and topographical maps of activity before (b) and after (a) artifact removal using ICA for subjects 1 (sub-01) and 2 (sub-02). All plots are created from preprocessed data by averaging over 320 trials for each subject. The 64 coloured waves correspond to the 64 EEG channels. The time axis shows the duration of a single trial and corresponds to a total duration of 4,000 ms. Notably, the time axis starts from a negative value of t = −1,000 ms, which corresponds to the 1,000 ms fixation period at the beginning of a trial. The times marked with arrows indicate the start of the fixation, inner-speech task, and rest periods (at −1,000 ms, 0 ms, and 2,000 ms, respectively).
Fig. 5 Plots of event-related potentials and topographical maps of activity before (b) and after (a) artifact removal using ICA for subjects 1 (sub-01) and 2 (sub-02). All plots are created from preprocessed data by averaging over 320 trials for each subject. The 64 coloured waves correspond to the 64 EEG channels. The time axis shows the duration of a single trial and corresponds to a total duration of 4,000 ms. Notably, the time axis starts from a negative value of t = −1,000 ms, which corresponds to the 1,000 ms fixation period at the beginning of a trial. The times marked with arrows indicate the start of the fixation, inner-speech task, and rest periods (at −1,000 ms, 0 ms, and 2,000 ms, respectively).CC BY · PubMed Central
Fig. 6 Plots of event-related potentials and topographical maps of activity before (b) and after (a) artifact removal using ICA for subjects 3 (sub-03) and 5 (sub-05). All plots are created from preprocessed data by averaging over 320 trials for each subject. The 64 coloured waves correspond to the 64 EEG channels. The time axis shows the duration of a single trial and corresponds to a total duration of 4,000 ms. Notably, the time axis starts from a negative value of t = −1,000 ms, which corresponds to the 1,000 ms fixation period at the beginning of a trial. The times marked with arrows indicate the start of the fixation, inner-speech task, and rest periods (at −1,000 ms, 0 ms, and 2,000 ms, respectively).
Fig. 6 Plots of event-related potentials and topographical maps of activity before (b) and after (a) artifact removal using ICA for subjects 3 (sub-03) and 5 (sub-05). All plots are created from preprocessed data by averaging over 320 trials for each subject. The 64 coloured waves correspond to the 64 EEG channels. The time axis shows the duration of a single trial and corresponds to a total duration of 4,000 ms. Notably, the time axis starts from a negative value of t = −1,000 ms, which corresponds to the 1,000 ms fixation period at the beginning of a trial. The times marked with arrows indicate the start of the fixation, inner-speech task, and rest periods (at −1,000 ms, 0 ms, and 2,000 ms, respectively).CC BY · PubMed Central
Fig. 7 Topological maps for subject 1 (sub-01) corresponding to stimuli from the number and social categories for components 19, 23, 35, and 42.
Fig. 7 Topological maps for subject 1 (sub-01) corresponding to stimuli from the number and social categories for components 19, 23, 35, and 42.CC BY · PubMed Central
Fig. 8 Topological maps for subject 1 (sub-01) for each stimulus in the number and social categories.
Fig. 8 Topological maps for subject 1 (sub-01) for each stimulus in the number and social categories.CC BY · PubMed Central
Fig. 9 Activation maps generated from a group-level fixed-effects model for the following comparisons: (a) areas more highly activated during the inner-speech task than the resting condition (number and social words combined) – this slice is selected to highlight activation in Broca’s area (coordinates: −42, 20, 20), and (b) areas more highly activated by social words than number words – this slice is selected to highlight activation in the secondary visual cortex (coordinates: 14, −92, −6). An increase in activation was found for the reverse contrast (i.e., increased activation for number words over social words).
Fig. 9 Activation maps generated from a group-level fixed-effects model for the following comparisons: (a) areas more highly activated during the inner-speech task than the resting condition (number and social words combined) – this slice is selected to highlight activation in Broca’s area (coordinates: −42, 20, 20), and (b) areas more highly activated by social words than number words – this slice is selected to highlight activation in the secondary visual cortex (coordinates: 14, −92, −6). An increase in activation was found for the reverse contrast (i.e., increased activation for number words over social words).CC BY · PubMed Central
Fig. 10 BOLD signals of each subject. (a) Area more highly activated during the inner-speech task in the rest condition (numbers and social words combined). (b) Areas more highly activated by social words than number words (no areas were more highly activated by number words than social words).
Fig. 10 BOLD signals of each subject. (a) Area more highly activated during the inner-speech task in the rest condition (numbers and social words combined). (b) Areas more highly activated by social words than number words (no areas were more highly activated by number words than social words).CC BY · PubMed Central
Fig. 11 The framewise displacement (FD; in mm) calculated across each subject and session is shown across each volume. The red dashed line indicates the voxel size of the functional images (2 × 2 × 2 mm).
Fig. 11 The framewise displacement (FD; in mm) calculated across each subject and session is shown across each volume. The red dashed line indicates the voxel size of the functional images (2 × 2 × 2 mm).CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 68 references

Research works printed but not linked: 1

  1. Penny, W. D., Friston, K. J., Ashburner, J. T., Kiebel, S. J. & Nichols, T. E. Statistical parametric mapping: the analysis of functional brain images (Elsevier, 2011).

Cited by

See all 35

Metadata

Authors
Simistira Liwicki, F., Gupta, V., Saini, R., De, K., Abid, N., Rakesh, S., Wellington, S., Wilson, H., Liwicki, M., & Eriksson, J.
Affiliations
  • Luleå University of Technology (SE)
  • University of Bath (GB)
  • Umeå University (SE)
Venue
Scientific Data
Published
2023-06-13
Volume
10
Issue
1
Pages
378-378
ISSN
2052-4463 (ISSN-L)
OpenAlex
W4380577383
Cited by
28
References
68
Citations by year
2026: 9 · 2025: 8 · 2024: 10 · 2023: 1
Type
data paper
Language
en
Open access
Licence
Locations
  • Scientific Data (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Epubl LTU (submittedVersion, other-oa)
  • KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
  • DiVA at Umeå University (Umeå University) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Grants for Excellent Research Projects Proposals of SRT.ai 2022
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
Keywords
EEG-fMRI fusion, brain-computer interface, word production, social words
MeSH terms
Brain, Electroencephalography, Humans, Magnetic Resonance Imaging, Speech, Speech Perception

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Bimodal electroencephalography-functional magnetic resonance imaging dataset for inner-speech recognition. https://app.cochat.ai/papers/10.1038/s41597-023-02286-w
The paper
Simistira Liwicki, F., Gupta, V., Saini, R., De, K., Abid, N., Rakesh, S., Wellington, S., Wilson, H., Liwicki, M., & Eriksson, J. (2023). Bimodal electroencephalography-functional magnetic resonance imaging dataset for inner-speech recognition. Scientific Data. https://doi.org/10.1038/s41597-023-02286-w

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.