Deep learning as a tool for neural data analysis: Speech classification and cross-frequency coupling in human sensorimotor cortex
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
A fundamental challenge in neuroscience is to understand what structure in the world is represented in spatially distributed patterns of neural activity from multiple single-trial measurements. This is often accomplished by learning a simple, linear transformations between neural features and features of the sensory stimuli or motor task. While successful in some early sensory processing areas, linear mappings are unlikely to be ideal tools for elucidating nonlinear, hierarchical representations of higher-order brain areas during complex tasks, such as the production of speech by humans. Here, we apply deep networks to predict produced speech syllables from a dataset of high gamma cortical surface electric potentials recorded from human sensorimotor cortex. We find that deep networks had higher decoding prediction accuracy compared to baseline models. Having established that deep networks extract more task relevant information from neural data sets relative to linear models (i.e., higher predictive accuracy), we next sought to demonstrate their utility as a data analysis tool for neuroscience. We first show that deep network's confusions revealed hierarchical latent structure in the neural data, which recapitulated the underlying articulatory nature of speech motor control. We next broadened the frequency features beyond high-gamma and identified a novel high-gamma-to-beta coupling during speech production. Finally, we used deep networks to compare task-relevant information in different neural frequency bands, and found that the high-gamma band contains the vast majority of information relevant for the speech prediction task, with little-to-no additional contribution from lower-frequency amplitudes. Together, these results demonstrate the utility of deep networks as a data analysis tool for basic and applied neuroscience.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
The origin of extracellular fields and currents — EEG, ECoG, LFP and spikes
Buzsáki, G., Anastassiou, C. A., & Koch, C. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3241 · Cited by 4,225 · cited 2×
“Being able to compare information across features is particularly useful for CSEPs which results from a variety of electrical processes in the brain [35].”
PENFIELD, W., & BOLDREY, E. (1937) · Brain · doi:10.1093/brain/60.4.389 · Cited by 4,188
Beta-band oscillations—Signalling the status quo?
Engel, A. K., & Fries, P. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.02.015 · Cited by 2,953
Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning
Alipanahi, B., Delong, A., Weirauch, M. T., & Frey, B. J. (2015) · Nature Biotechnology · doi:10.1038/nbt.3300 · Cited by 2,873
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
Learning to control a brain–machine interface for reaching and grasping by primates
Carmena, J. M., Lebedev, M. A., Crist, R. E., O'Doherty, J. E., Santucci, D. M., Dimitrov, D. F., Patil, P. G., Henriquez, C. S., & Nicolelis, M. A. L. (2003) · PLoS Biology · doi:10.1371/journal.pbio.0000042 · Cited by 1,761 · cited 1×
“Since deep networks are highly parameterized nonlinear models, their online interactions with learning may be more complex than typical methods [63].”
Different Origins of Gamma Rhythm and High-Gamma Activity in Macaque Visual Cortex
Ray, S., & Maunsell, J. H. R. (2011) · PLoS Biology · doi:10.1371/journal.pbio.1000610 · Cited by 1,012
Power-Law Scaling in the Brain Surface Electric Potential
Miller, K. J., Sorensen, L. B., Ojemann, J. G., & den Nijs, M. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000609 · Cited by 742
Michalareas, G., Vezoli, J., van Pelt, S., Schoffelen, J.-M., Kennedy, H., & Fries, P. (2016) · Neuron · doi:10.1016/j.neuron.2015.12.018 · Cited by 696
Cited by
A comprehensive survey on applications of transformers for deep learning tasks
Islam, S., Elmekki, H., Elsebai, A., Bentahar, J., Drawel, N., Rjoub, G., & Pedrycz, W. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122666 · Cited by 487
Speech synthesis from ECoG using densely connected 3D convolutional neural networks
Angrick, M., Herff, C., Mugler, E., Tate, M. C., Slutzky, M. W., Krusienski, D. J., & Schultz, T. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab0c59 · Cited by 225
“To extract meaningful information from the ECoG signals, we calculated logarithmic power in the broadband-gamma frequency range (70–170 Hz, filter order 13 and 14, respectively), which is thought to largely reflect ensemble spiking [48] and contains localized information about movement and speech processes [49].”
MethodImagined speech can be decoded from low- and cross-frequency intracranial EEG features
Proix, T., Delgado Saa, J., Christen, A., Martin, S., Pasley, B. N., Knight, R. T., Tian, X., Poeppel, D., Doyle, W. K., Devinsky, O., et al. (2022) · Nature Communications · doi:10.1038/s41467-021-27725-3 · Cited by 212
Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis
Stavisky, S. D., Willett, F. R., Wilson, G. H., Murphy, B. A., Rezaii, P., Avansino, D. T., Memberg, W. D., Miller, J. P., Kirsch, R. F., Hochberg, L. R., et al. (2019) · eLife · doi:10.7554/elife.46015 · Cited by 105
“This observation is consistent with previous ECoG studies (Bouchard et al., 2013; Cheung et al., 2016; Livezey et al., 2019; Moses et al., 2019; Mugler et al., 2014), although the larger neural differences we observed between unvoiced k and p and the beginning of their voiced counterparts at the start of ga and ba sug…”
BackgroundResultDecoding spoken English from intracortical electrode arrays in dorsal precentral gyrus
Wilson, G. H., Stavisky, S. D., Willett, F. R., Avansino, D. T., Kelemen, J. N., Hochberg, L. R., Henderson, J. M., Druckmann, S., & Shenoy, K. V. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abbfef · Cited by 103
Metadata
- Authors
- Livezey, J. A., Bouchard, K. E., & Chang, E. F.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-09-16
- Volume
- 15
- Issue
- 9
- Pages
- e1007091
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2794504802
- Cited by
- 58
- References
- 20
- FWCI
- 4.73
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Deep learning as a tool for neural data analysis: Speech classification and cross-frequency coupling in human sensorimotor cortex. https://app.cochat.ai/papers/10.1371/journal.pcbi.1007091
- The paper
- Livezey, J. A., Bouchard, K. E., & Chang, E. F. (2019). Deep learning as a tool for neural data analysis: Speech classification and cross-frequency coupling in human sensorimotor cortex. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1007091
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.