Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.

Perceptual restoration of masked speech in human cortex

Open access

Abstract

Humans are adept at understanding speech despite the fact that our natural listening environment is often filled with interference. An example of this capacity is phoneme restoration, in which part of a word is completely replaced by noise, yet listeners report hearing the whole word. The neurological basis for this unconscious fill-in phenomenon is unknown, despite being a fundamental characteristic of human hearing. Here, using direct cortical recordings in humans, we demonstrate that missing speech is restored at the acoustic-phonetic level in bilateral auditory cortex, in real-time. This restoration is preceded by specific neural activity patterns in a separate language area, left frontal cortex, which predicts the word that participants later report hearing. These results demonstrate that during speech perception, missing acoustic content is synthesized online from the integration of incoming sensory cues and the internal neural dynamics that bias word-level expectation and prediction.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/ncomms13619

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Stimuli and single electrode online phoneme restoration effects. (a,b) Participants listened to pairs of spoken words (/fæstr/ (a) versus/fæktr/ (b)) that were acoustically identical except for a critical phoneme that differentiated their meaning (vertical solid and second dashed lines; first dashed line is word onset). (c) The critical phoneme was also replaced by broadband noise (/fæ#tr/), and on each trial, participants reported which word they heard. (d) Behavioural results show bistable perception on noise trials. (e) Location of representative posterior STG electrode in f. (f) STG electrode shows selectivity for /s/ compared to /k/ (solid blue line stronger response than solid red line immediately after critical phoneme, unshaded region). Trials were sorted depending on which word participants perceived. Responses to noise stimuli were similar to the original version of the perceived phoneme (dotted lines; *signifies 99% CIs only overlapping for same coloured curves; shaded error±s.e.m. across trials). (g) RI describes the magnitude of neural restoration as the relative distances between each noise and original pair in f. When the dotted line is in the region shaded with the same colour, the electrode's activity reflects the participant's percept. (h) Across all participants, word pairs and electrodes, the magnitude of the difference between RI values illustrates that when these neural populations differentiate original stimuli, they also differentiate noise trials, beginning at the onset of the critical phoneme (red bar, one-way t tests, P<0.05, Bonferroni corrected). Shaded error±s.e.m. across word pairs.
Figure 1 Stimuli and single electrode online phoneme restoration effects. (a,b) Participants listened to pairs of spoken words (/fæstr/ (a) versus/fæktr/ (b)) that were acoustically identical except for a critical phoneme that differentiated their meaning (vertical solid and second dashed lines; first dashed line is word onset). (c) The critical phoneme was also replaced by broadband noise (/fæ#tr/), and on each trial, participants reported which word they heard. (d) Behavioural results show bistable perception on noise trials. (e) Location of representative posterior STG electrode in f. (f) STG electrode shows selectivity for /s/ compared to /k/ (solid blue line stronger response than solid red line immediately after critical phoneme, unshaded region). Trials were sorted depending on which word participants perceived. Responses to noise stimuli were similar to the original version of the perceived phoneme (dotted lines; *signifies 99% CIs only overlapping for same coloured curves; shaded error±s.e.m. across trials). (g) RI describes the magnitude of neural restoration as the relative distances between each noise and original pair in f. When the dotted line is in the region shaded with the same colour, the electrode's activity reflects the participant's percept. (h) Across all participants, word pairs and electrodes, the magnitude of the difference between RI values illustrates that when these neural populations differentiate original stimuli, they also differentiate noise trials, beginning at the onset of the critical phoneme (red bar, one-way t tests, P<0.05, Bonferroni corrected). Shaded error±s.e.m. across word pairs.CC BY · PubMed Central
Figure 2 Stimuli and single electrode online phoneme restoration effects for a representative word pair where the participant did not show bistable perception. (a,b) Subjects listened to pairs of spoken words (/wƅkǝrz/ (a) versus/ wƅtǝrz/ (b)) that were acoustically identical except for a critical phoneme that differentiated their meaning (vertical solid and second dashed lines; first dashed line is word onset). (c) The critical phoneme was also replaced by broadband noise (/wƅ#ǝrz/), and on each trial subjects reported which word they heard. (d) Behavioural results showed that the noise was always perceived as /t/. (e) Location of representative STG electrode in f. Data are from the same subject as in Fig. 1. (f) Single representative left hemisphere STG electrode shows selectivity for /k/ compared with /t/ (solid blue line stronger response than solid red line immediately after critical phoneme, unshaded region). Responses to noise stimuli were similar to the original version of /t/ (dotted red line; *signifies 99% CIs only overlapping for red curves; shaded error±s.e.m. across trials). (g) RI describes the magnitude of neural restoration as the relative distances between each noise and original pair in f. When the dotted line is in the region shaded with the same colour, the electrode's activity reflects the subject's percept. (h) Across all word pairs that did not exhibit bistable perception, the average timecourse of the RI metric for all electrodes shows neural restoration effects beginning ∼150 ms after critical phoneme onset (red bar: one-way t test against baseline, P<0.05, false discovery rate corrected for time points). Shaded error±s.e.m. across word pairs.
Figure 2 Stimuli and single electrode online phoneme restoration effects for a representative word pair where the participant did not show bistable perception. (a,b) Subjects listened to pairs of spoken words (/wƅkǝrz/ (a) versus/ wƅtǝrz/ (b)) that were acoustically identical except for a critical phoneme that differentiated their meaning (vertical solid and second dashed lines; first dashed line is word onset). (c) The critical phoneme was also replaced by broadband noise (/wƅ#ǝrz/), and on each trial subjects reported which word they heard. (d) Behavioural results showed that the noise was always perceived as /t/. (e) Location of representative STG electrode in f. Data are from the same subject as in Fig. 1. (f) Single representative left hemisphere STG electrode shows selectivity for /k/ compared with /t/ (solid blue line stronger response than solid red line immediately after critical phoneme, unshaded region). Responses to noise stimuli were similar to the original version of /t/ (dotted red line; *signifies 99% CIs only overlapping for red curves; shaded error±s.e.m. across trials). (g) RI describes the magnitude of neural restoration as the relative distances between each noise and original pair in f. When the dotted line is in the region shaded with the same colour, the electrode's activity reflects the subject's percept. (h) Across all word pairs that did not exhibit bistable perception, the average timecourse of the RI metric for all electrodes shows neural restoration effects beginning ∼150 ms after critical phoneme onset (red bar: one-way t test against baseline, P<0.05, false discovery rate corrected for time points). Shaded error±s.e.m. across word pairs.CC BY · PubMed Central
Figure 3 Stimulus spectrogram reconstruction reveals warping of noise to perceived phoneme. (a,b) Acoustic spectrograms for a representative word pair (/fæstr/, (a), versus /fæktr/, (b)) differ primarily in the presence of a high-frequency component during the critical phoneme in a (green arrow). (c,d) Spectrograms from (a,b) reconstructed from electrode population activity show that the high-frequency component is present in / fæstr/ (c, green arrow) and absent in /fæktr/ (d). (e,f) Spectrogram reconstruction of noise trials was divided according to which word the participant heard on each trial. During the critical phoneme, a high-frequency component is visible only for trials perceived as /fæstr/ (e, green arrow) and not for /fæktr/ (f). (g) Power spectra of the critical phoneme for c–f show close correspondence between noise and original phonemes, particularly in mid-high frequencies.
Figure 3 Stimulus spectrogram reconstruction reveals warping of noise to perceived phoneme. (a,b) Acoustic spectrograms for a representative word pair (/fæstr/, (a), versus /fæktr/, (b)) differ primarily in the presence of a high-frequency component during the critical phoneme in a (green arrow). (c,d) Spectrograms from (a,b) reconstructed from electrode population activity show that the high-frequency component is present in / fæstr/ (c, green arrow) and absent in /fæktr/ (d). (e,f) Spectrogram reconstruction of noise trials was divided according to which word the participant heard on each trial. During the critical phoneme, a high-frequency component is visible only for trials perceived as /fæstr/ (e, green arrow) and not for /fæktr/ (f). (g) Power spectra of the critical phoneme for c–f show close correspondence between noise and original phonemes, particularly in mid-high frequencies.CC BY · PubMed Central
Figure 4 Timecourse of stimulus classification shows pre-stimulus frontal lobe bias for restored phonemes. (a) Trials were classified using population neural activity and compared with reported perception. Original word classification accuracy peaked ∼200 ms after critical phoneme onset (black line, blue arrow). Noise trial classification accuracy was similar, and showed above-chance classification before critical phoneme onset (green line; orange arrow). Shaded error±s.e.m. across word pairs. (b–e) Classification weights for all subjects mapped onto a common cortical surface (MNI). During the pre-critical phoneme period, classification performance was driven by bilateral superior temporal cortex for original (b) and noise (c) trials. Noise trials also showed significantly greater weights in left inferior frontal cortex compared with original trials (orange box). During the post-critical phoneme period, classification performance was driven by bilateral superior temporal cortex for original (d) and noise (e) trials, with greater weights in left superior temporal cortex for original trials (blue box).
Figure 4 Timecourse of stimulus classification shows pre-stimulus frontal lobe bias for restored phonemes. (a) Trials were classified using population neural activity and compared with reported perception. Original word classification accuracy peaked ∼200 ms after critical phoneme onset (black line, blue arrow). Noise trial classification accuracy was similar, and showed above-chance classification before critical phoneme onset (green line; orange arrow). Shaded error±s.e.m. across word pairs. (b–e) Classification weights for all subjects mapped onto a common cortical surface (MNI). During the pre-critical phoneme period, classification performance was driven by bilateral superior temporal cortex for original (b) and noise (c) trials. Noise trials also showed significantly greater weights in left inferior frontal cortex compared with original trials (orange box). During the post-critical phoneme period, classification performance was driven by bilateral superior temporal cortex for original (d) and noise (e) trials, with greater weights in left superior temporal cortex for original trials (blue box).CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 170

Metadata

Authors
Leonard, M. K., Baud, M. O., Sjerps, M. J., & Chang, E. F.
Affiliations
  • University of California, San Francisco (US)
  • Radboud University Nijmegen (NL)
Corresponding author
Chang, Edward F.
Venue
Nature Communications
Published
2016-12-20
Volume
7
Issue
1
Pages
13619-13619
ISSN
2041-1723 (ISSN-L)
OpenAlex
W2563866812
Cited by
170
References
49
Citations by year
2026: 8 · 2025: 13 · 2024: 16 · 2023: 37 · 2022: 15 · 2021: 15 · 2020: 17 · 2019: 19 · 2018: 20 · 2017: 10
FWCI
10.29
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • MPG.PuRe (Max Planck Society) (submittedVersion, cc-by)
  • Radboud Repository (Radboud University) (submittedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • eScholarship (California Digital Library) (submittedVersion)
  • Radboud Repository (Radboud University) (submittedVersion)
Funders
  • New York Stem Cell Foundation
  • McKnight Foundation
  • Shurl and Kay Curci Foundation
  • European Commission (623072)
  • National Institutes of Health (DP2-OD00862, OD00862, R00-NS065120, 5r00ns065120-04)
  • National Institute on Deafness and Other Communication Disorders (F32DC013486, R01DC012379)
  • National Institute of Neurological Disorders and Stroke (R00-NS065120)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience and Music Perception
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Phonetics and Phonology Research
Keywords
auditory cortex, speech perception, language areas, left frontal cortex, word prediction, neural dynamics, sensory cue integration
MeSH terms
Acoustic Stimulation, Adult, Auditory Cortex, Female, Humans, Male, Middle Aged, Noise, Perceptual Masking, Phonetics, Speech Perception

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.