CoChat

Prediction Errors but Not Sharpened Signals Simulate Multivoxel fMRI Patterns during Speech Perception

Open access
Blank, Helen, Davis, Matthew H. · 2016 · PLOS Biology

The DOI matches its Crossref record; 114 of the 114 references it cites were found in a registry or an index.

Abstract

Successful perception depends on combining sensory input with prior knowledge. However, the underlying mechanism by which these two sources of information are combined is unknown. In speech perception, as in other domains, two functionally distinct coding schemes have been proposed for how expectations influence representation of sensory evidence. Traditional models suggest that expected features of the speech input are enhanced or sharpened via interactive activation (Sharpened Signals). Conversely, Predictive Coding suggests that expected features are suppressed so that unexpected features of the speech input (Prediction Errors) are processed further. The present work is aimed at distinguishing between these two accounts of how prior knowledge influences speech perception. By combining behavioural, univariate, and multivariate fMRI measures of how sensory detail and prior expectations influence speech perception with computational modelling, we provide evidence in favour of Prediction Error computations. Increased sensory detail and informative expectations have additive behavioural and univariate neural effects because they both improve the accuracy of word report and reduce the BOLD signal in lateral temporal lobe regions. However, sensory detail and informative expectations have interacting effects on speech representations shown by multivariate fMRI in the posterior superior temporal sulcus. When prior knowledge was absent, increased sensory detail enhanced the amount of speech information measured in superior temporal multivoxel patterns, but with informative expectations, increased sensory detail reduced the amount of measured information. Computational simulations of Sharpened Signals and Prediction Errors during speech perception could both explain these behavioural and univariate fMRI observations. However, the multivariate fMRI observations were uniquely simulated by a Prediction Error and not a Sharpened Signal model. The interaction between prior expectation and sensory detail provides evidence for a Predictive Coding account of speech perception. Our work establishes methods that can be used to distinguish representations of Prediction Error and Sharpened Signals in other perceptual domains.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pbio.1002577

Cites

114 of 114 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig 1 Two computational models for how matching or neutral prior expectations influence processing of sensory signals at different levels of clarity: (A) Sharpened Signal model and (B) Prediction Error model. For both accounts, neural representations are derived by combining the sensory input with prior expectation. However, the underlying computations and information content in neural representations differ. (A) Sharpened Signal model: Prior expectation is used to multiply sensory input, leading to more specific representations for expected compared to unexpected sensory input (Sharpened Signals, SS). This leads to additive effects of sensory detail and matching prior expectation on the information content of neural representations. (B) Prediction Error model: Prior expectation is subtracted from the sensory input such that neural representations encode the difference between expected and actual input (Prediction Error, PE). This leads to an interaction between sensory detail and prior expectations, with most informative neural representations found when clearer signals follow neutral expectations, or when degraded signals match informative prior expectations. Critically, when clear signals match informative prior expectations, this produces a small and uninformative Prediction Error (Match 12-channel condition). The information content of neural representations (y-axis) contained in SS (A) and in PE (B) refers to the signal that is passed forward after the input and prior have been combined (bottom bars). This allows us to test which of these neural representations best describes measured fMRI pattern information. In each model, neural activity patterns are represented by greyscale values over sets of units. Negative Prediction Error values are shown with a white outline.
Fig 1 Two computational models for how matching or neutral prior expectations influence processing of sensory signals at different levels of clarity: (A) Sharpened Signal model and (B) Prediction Error model. For both accounts, neural representations are derived by combining the sensory input with prior expectation. However, the underlying computations and information content in neural representations differ. (A) Sharpened Signal model: Prior expectation is used to multiply sensory input, leading to more specific representations for expected compared to unexpected sensory input (Sharpened Signals, SS). This leads to additive effects of sensory detail and matching prior expectation on the information content of neural representations. (B) Prediction Error model: Prior expectation is subtracted from the sensory input such that neural representations encode the difference between expected and actual input (Prediction Error, PE). This leads to an interaction between sensory detail and prior expectations, with most informative neural representations found when clearer signals follow neutral expectations, or when degraded signals match informative prior expectations. Critically, when clear signals match informative prior expectations, this produces a small and uninformative Prediction Error (Match 12-channel condition). The information content of neural representations (y-axis) contained in SS (A) and in PE (B) refers to the signal that is passed forward after the input and prior have been combined (bottom bars). This allows us to test which of these neural representations best describes measured fMRI pattern information. In each model, neural activity patterns are represented by greyscale values over sets of units. Negative Prediction Error values are shown with a white outline.CC BY · PubMed Central
Fig 2 Design and experimental conditions. We used sparse imaging to record fMRI responses while participants see written words, hear subsequently presented degraded spoken words, and say what word they heard or read previously. We used two levels of sensory detail (4- and 12-channel) for presentation of the spoken words and conditions containing different pairings of written and spoken words: (1) matching written text + spoken words (“SING” + sing); (2) neutral written text (“XXXX”) + spoken words (e.g., fork); and (3) written-only text (“PASS”). Following 1/6 of all trials, participants were cued with a question mark to say aloud the previous written or spoken word. In addition, we inserted fixation crosses, null events, and trials in which written text partially or totally mismatched with spoken words (see Materials and Methods for details).
Fig 2 Design and experimental conditions. We used sparse imaging to record fMRI responses while participants see written words, hear subsequently presented degraded spoken words, and say what word they heard or read previously. We used two levels of sensory detail (4- and 12-channel) for presentation of the spoken words and conditions containing different pairings of written and spoken words: (1) matching written text + spoken words (“SING” + sing); (2) neutral written text (“XXXX”) + spoken words (e.g., fork); and (3) written-only text (“PASS”). Following 1/6 of all trials, participants were cued with a question mark to say aloud the previous written or spoken word. In addition, we inserted fixation crosses, null events, and trials in which written text partially or totally mismatched with spoken words (see Materials and Methods for details).CC BY · PubMed Central
Fig 3 Comparison of behavioural and univariate fMRI results with model output. (A) Behavioural results. Matching expectations and increased sensory detail improved perception of degraded spoken words. (B) Univariate results. Mean beta values extracted from the posterior STS (pSTS, MNI: x = -52, y = -38, z = 6) show reduced BOLD signal during Match conditions (solid) in contrast to Neutral conditions (open). Error bars for the empirical data indicate standard error of the mean after between-subject variability has been removed, which is appropriate for repeated-measures comparisons [62]. (C) Main effect of prior expectations rendered on a canonical brain (p < 0.05 voxelwise FWE, n = 21). White circle marks the region of interest in the posterior STS. (D/E) Sharpened Signal model (orange) and (F/G) Prediction Error model (blue). For comparison with the behavioural results (D/F) we assessed word recognition accuracy in the model based on the final lexical representation (i.e., which word the model selected as presented), and for comparison with the univariate results (E/G) we assessed the number of activation updates required to reach the stopping criterion. Error bars for both simulations indicate the standard error of the mean over 1,000 replications. Please refer to S1 Data at https://osf.io/2ze9n/ (doi: 10.17605/OSF.IO/2ZE9N) for the numerical values underlying these figures.
Fig 3 Comparison of behavioural and univariate fMRI results with model output. (A) Behavioural results. Matching expectations and increased sensory detail improved perception of degraded spoken words. (B) Univariate results. Mean beta values extracted from the posterior STS (pSTS, MNI: x = -52, y = -38, z = 6) show reduced BOLD signal during Match conditions (solid) in contrast to Neutral conditions (open). Error bars for the empirical data indicate standard error of the mean after between-subject variability has been removed, which is appropriate for repeated-measures comparisons [62]. (C) Main effect of prior expectations rendered on a canonical brain (p < 0.05 voxelwise FWE, n = 21). White circle marks the region of interest in the posterior STS. (D/E) Sharpened Signal model (orange) and (F/G) Prediction Error model (blue). For comparison with the behavioural results (D/F) we assessed word recognition accuracy in the model based on the final lexical representation (i.e., which word the model selected as presented), and for comparison with the univariate results (E/G) we assessed the number of activation updates required to reach the stopping criterion. Error bars for both simulations indicate the standard error of the mean over 1,000 replications. Please refer to S1 Data at https://osf.io/2ze9n/ (doi: 10.17605/OSF.IO/2ZE9N) for the numerical values underlying these figures.CC BY · PubMed Central
Fig 4 Multivariate fMRI results and simulation. (A) Hypothesized representational dissimilarity matrices. These four matrices were used to test similarity between words that share vowels within each of the four critical conditions (Match 4-channel, Neutral 4-channel, Match 12-channel, and Neutral 12-channel). Similarity between responses to identical items (on the main diagonal) was excluded, as was similarity between items in different conditions (“Not a Number” [NaN] values depicted in grey). Similarity between items containing the same vowel was predicted (zeroes in blue), whereas items containing different vowels were predicted to have more dissimilar representations (ones in red). These matrices are correlated with observed and simulated representational similarity. (B) RSA results. Fisher-z-transformed Spearman correlation coefficients for each of the four conditions in the left posterior STS (extracted from an independent ROI, [57]) show a significant interaction between sensory detail and prior expectation. Error bars indicate standard error of the mean after between-subject variability has been removed, which is appropriate for repeated-measures comparisons [62]. (C,D) Model comparison. Fisher-z-transformed Spearman correlation coefficients for each of the four conditions in the two models. (C) Sharpened Signal model (in orange) shows that both prior knowledge and sensory detail increase similarity for words that share the same vowel. (D) Prediction Error model (in blue) shows opposite effects of sensory detail in neutral and matching prior knowledge conditions, consistent with the RSA results (B). Error bars in (C) and (D) indicate standard error of the mean over 1,000 replications of these simulations. Please refer to S1 Data at https://osf.io/2ze9n/ (doi: 10.17605/OSF.IO/2ZE9N) for the numerical values underlying these figures.
Fig 4 Multivariate fMRI results and simulation. (A) Hypothesized representational dissimilarity matrices. These four matrices were used to test similarity between words that share vowels within each of the four critical conditions (Match 4-channel, Neutral 4-channel, Match 12-channel, and Neutral 12-channel). Similarity between responses to identical items (on the main diagonal) was excluded, as was similarity between items in different conditions (“Not a Number” [NaN] values depicted in grey). Similarity between items containing the same vowel was predicted (zeroes in blue), whereas items containing different vowels were predicted to have more dissimilar representations (ones in red). These matrices are correlated with observed and simulated representational similarity. (B) RSA results. Fisher-z-transformed Spearman correlation coefficients for each of the four conditions in the left posterior STS (extracted from an independent ROI, [57]) show a significant interaction between sensory detail and prior expectation. Error bars indicate standard error of the mean after between-subject variability has been removed, which is appropriate for repeated-measures comparisons [62]. (C,D) Model comparison. Fisher-z-transformed Spearman correlation coefficients for each of the four conditions in the two models. (C) Sharpened Signal model (in orange) shows that both prior knowledge and sensory detail increase similarity for words that share the same vowel. (D) Prediction Error model (in blue) shows opposite effects of sensory detail in neutral and matching prior knowledge conditions, consistent with the RSA results (B). Error bars in (C) and (D) indicate standard error of the mean over 1,000 replications of these simulations. Please refer to S1 Data at https://osf.io/2ze9n/ (doi: 10.17605/OSF.IO/2ZE9N) for the numerical values underlying these figures.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 114 references

Cited by

See all 181

Metadata

Authors
Blank, H., & Davis, M. H.
Affiliations
MRC Cognition and Brain Sciences Unit (GB)
Corresponding author
Blank, Helen
Venue
PLOS Biology
Published
2016-11-15
Volume
14
Issue
11
Pages
e1002577
ISSN
  • 1544-9173 (ISSN-L)
  • 1545-7885
OpenAlex
W2549771691
Cited by
181
References
112
Citations by year
2026: 10 · 2025: 15 · 2024: 21 · 2023: 14 · 2022: 55 · 2021: 14 · 2020: 18 · 2019: 14 · 2018: 13 · 2017: 7
FWCI
7.21
Type
journal article
Language
en
Open access
Locations
  • PLoS Biology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • OSF Preprints (OSF Preprints) (submittedVersion)
Funders
Medical Research Council (MC-A060-5PQ80)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Phonetics and Phonology Research
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience and Music Perception
Keywords
speech perception, prediction error, predictive coding, interactive activation, prior expectations, sensory detail, posterior superior temporal sulcus, lateral temporal lobe, perceptual representation
MeSH terms
Behavior, Humans, Magnetic Resonance Imaging, Models, Theoretical, Speech Perception, Temporal Lobe, Multivariate Analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Prediction Errors but Not Sharpened Signals Simulate Multivoxel fMRI Patterns during Speech Perception. https://app.cochat.ai/papers/10.1371/journal.pbio.1002577
The paper
Blank, H., & Davis, M. H. (2016). Prediction Errors but Not Sharpened Signals Simulate Multivoxel fMRI Patterns during Speech Perception. PLOS Biology. https://doi.org/10.1371/journal.pbio.1002577

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.