CoChat

The signal processing architecture underlying subjective reports of sensory awareness

Open access
Maniscalco, Brian, Lau, Hakwan · 2016 · Neuroscience of Consciousness

The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.

Abstract

What is the relationship between perceptual information processing and subjective perceptual experience? Empirical dissociations between stimulus identification performance and subjective reports of stimulus visibility are crucial for shedding light on this question. We replicated a finding that metacontrast masking can produce such a dissociation (Lau and Passingham, 2006), and report a novel finding that this paradigm can also dissociate stimulus identification performance from the efficacy with which visibility ratings predict task performance. We explored various hypotheses about the relationship between perceptual task performance and visibility rating by implementing them in computational models and using formal model comparison techniques to assess which ones best captured the unusual patterns in the data. The models fell into three broad categories: Single Channel models, which hold that task performance and visibility ratings are based on the same underlying source of information; Dual Channel models, which hold that there are two independent processing streams that differentially contribute to task performance and visibility rating; and Hierarchical models, which hold that a late processing stage generates visibility ratings by evaluating the quality of early perceptual processing. Taking into account the quality of data fitting and model complexity, we found that Hierarchical models perform best at capturing the observed behavioral dissociations. Because current theories of visual awareness map well onto these different model structures, a formal comparison between them is a powerful approach for arbitrating between the different theories.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/nc/niw002

Cites

69 of 69 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1. Schematic diagram for the three categories of models. (Left) According to a Single Channel model, the same process gives rise to both objective judgments (e.g. perceptual decisions about the identity of a stimulus) and subjective judgments (e.g. confidence ratings or visibility ratings). The model can still support some independence between task performance and subjective reports by supposing that the sensory evidence is a continuous variable that can be evaluated by setting various decision criteria (Fig. 2; Macmillan and Creelman, 2005). (Middle) An alternative model is that objective and subjective judgments are driven by two parallel processes, each influenced by independent sources of noise. Differential contribution of the two channels to objective and subjective judgments can lead to dissociations between the two kinds of responses. Note that the model can allow that each channel can contribute both kinds of judgments to some extent. In particular, one would expect that the channel which primarily influences one’s subjective ratings would also heavily influence one’s objective task response. For instance, when an observer subjectively reports clearly and vividly seeing squares, this should strongly correlate with objective judgments that the stimuli on the current trial are squares. (Right) Another alternative is that objective and subjective judgments are driven by different processes that are organized in a serial hierarchy, such that an early stage of processing generates the objective judgment and a later stage of processing generates the subjective judgment, as if the latter evaluates the quality of the former. Note that on this model, the second stage inherits the noise of the first stage, and thus the two are not entirely independent. However, the influence is one sided; the “subjective” stage does not influence the “objective” stage of processing
Figure 1. Schematic diagram for the three categories of models. (Left) According to a Single Channel model, the same process gives rise to both objective judgments (e.g. perceptual decisions about the identity of a stimulus) and subjective judgments (e.g. confidence ratings or visibility ratings). The model can still support some independence between task performance and subjective reports by supposing that the sensory evidence is a continuous variable that can be evaluated by setting various decision criteria (Fig. 2; Macmillan and Creelman, 2005). (Middle) An alternative model is that objective and subjective judgments are driven by two parallel processes, each influenced by independent sources of noise. Differential contribution of the two channels to objective and subjective judgments can lead to dissociations between the two kinds of responses. Note that the model can allow that each channel can contribute both kinds of judgments to some extent. In particular, one would expect that the channel which primarily influences one’s subjective ratings would also heavily influence one’s objective task response. For instance, when an observer subjectively reports clearly and vividly seeing squares, this should strongly correlate with objective judgments that the stimuli on the current trial are squares. (Right) Another alternative is that objective and subjective judgments are driven by different processes that are organized in a serial hierarchy, such that an early stage of processing generates the objective judgment and a later stage of processing generates the subjective judgment, as if the latter evaluates the quality of the former. Note that on this model, the second stage inherits the noise of the first stage, and thus the two are not entirely independent. However, the influence is one sided; the “subjective” stage does not influence the “objective” stage of processingCC BY-NC · PubMed Central
Figure 2. The standard SDT model. All models under consideration are built upon the foundation of the standard SDT model. This model assumes that stimulus categories S1 and S2 each generate normal distributions of perceptual evidence along an internal decision axis. The observer segments the decision axis into discrete regions using a type 1 criterion (for making a stimulus classification response) and a set of type 2 criteria (for rating subjective levels of decision confidence or percept visibility). The stimulus classification and subjective rating reported by the observer on any given trial are determined by which region of the decision axis contains the perceptual evidence observed on that trial, as illustrated in the figure. The probability with which the observer produces a given (response, visibility) pair upon being shown stimulus SN is equal to the area under the curve f(x|SN) in the region of the decision axis corresponding to that response pair. Note that it need not be the case that the type 1 criterion is located at the intersection of the distributions, or that the type 2 criteria are symmetrically distributed around the type 1 criterion
Figure 2. The standard SDT model. All models under consideration are built upon the foundation of the standard SDT model. This model assumes that stimulus categories S1 and S2 each generate normal distributions of perceptual evidence along an internal decision axis. The observer segments the decision axis into discrete regions using a type 1 criterion (for making a stimulus classification response) and a set of type 2 criteria (for rating subjective levels of decision confidence or percept visibility). The stimulus classification and subjective rating reported by the observer on any given trial are determined by which region of the decision axis contains the perceptual evidence observed on that trial, as illustrated in the figure. The probability with which the observer produces a given (response, visibility) pair upon being shown stimulus SN is equal to the area under the curve f(x|SN) in the region of the decision axis corresponding to that response pair. Note that it need not be the case that the type 1 criterion is located at the intersection of the distributions, or that the type 2 criteria are symmetrically distributed around the type 1 criterionCC BY-NC · PubMed Central
Figure 3. Experimental design and behavioral results. (A) We used a paradigm based on metacontrast masking, similar to the one used in a previous study (Lau and Passingham, 2006). In every trial, the subject was presented with a set of squares or diamonds (i.e. tilted squares). After a varying SOA (the temporal gap between the two sets of stimuli), a metacontrast mask was presented at the location of each shape. The masks were not drawn “on top of” the targets, but rather only traced the spatial contours where the targets had been presented. Nevertheless, the masks were successful in impairing target visibility. In each trial, subjects first decided whether the targets were squares or diamonds, and then gave subjective visibility ratings (four levels) to indicate how clearly they saw the identity of the targets. (B) Replicating previous findings (Lau and Passingham, 2006), this masking procedure gives rise to a U-shaped masking function when stimulus identification performance is plotted against SOA. The average level of subjective visibility ratings across SOAs, however, did not take the same shape, and reflected a bias toward giving lower ratings at lower SOAs. Shown here are data from a group of subjects (n = 20) who exhibited this dissociation particularly strongly. Right panel plots the same data as left panel, now showing visibility as a function of percent correct, to illustrate the dissociation more clearly. Data points correspond to SOA from the left panel. Visibility is an approximately linear function of percent correct, with the notable exception of SOA = 16 ms. This SOA had similar percent correct as SOA = 100 ms and 116.7 ms, but much lower visibility. (C) Data from all 39 subjects selected for model comparison analysis. (D) Data from the 19 subjects exhibiting a relatively weaker dissociation
Figure 3. Experimental design and behavioral results. (A) We used a paradigm based on metacontrast masking, similar to the one used in a previous study (Lau and Passingham, 2006). In every trial, the subject was presented with a set of squares or diamonds (i.e. tilted squares). After a varying SOA (the temporal gap between the two sets of stimuli), a metacontrast mask was presented at the location of each shape. The masks were not drawn “on top of” the targets, but rather only traced the spatial contours where the targets had been presented. Nevertheless, the masks were successful in impairing target visibility. In each trial, subjects first decided whether the targets were squares or diamonds, and then gave subjective visibility ratings (four levels) to indicate how clearly they saw the identity of the targets. (B) Replicating previous findings (Lau and Passingham, 2006), this masking procedure gives rise to a U-shaped masking function when stimulus identification performance is plotted against SOA. The average level of subjective visibility ratings across SOAs, however, did not take the same shape, and reflected a bias toward giving lower ratings at lower SOAs. Shown here are data from a group of subjects (n = 20) who exhibited this dissociation particularly strongly. Right panel plots the same data as left panel, now showing visibility as a function of percent correct, to illustrate the dissociation more clearly. Data points correspond to SOA from the left panel. Visibility is an approximately linear function of percent correct, with the notable exception of SOA = 16 ms. This SOA had similar percent correct as SOA = 100 ms and 116.7 ms, but much lower visibility. (C) Data from all 39 subjects selected for model comparison analysis. (D) Data from the 19 subjects exhibiting a relatively weaker dissociationCC BY-NC · PubMed Central
Figure 4. Model fits for task performance and reported visibility. Three categories of models (Single Channel, Dual Channel, and Hierarchical) were fitted to the behavioral data from the metacontrast masking paradigm. We tested multiple versions of each category of model (see “Materials and Methods” section for details). Shown here are the best-fitting models from each category, selected according to formal model comparison techniques (Fig. 6). The Hierarchical model performed best at capturing the dissociation between task performance and reported levels of stimulus visibility. This dissociation is made readily apparent by plotting visibility reports against task performance, as depicted in the bottom row of figures; the relationship is not monotonic, but exhibits a sharp spike at around 80–85% correct, reflecting that short SOAs had lower visibility than long SOAs in spite of having similar task performance
Figure 4. Model fits for task performance and reported visibility. Three categories of models (Single Channel, Dual Channel, and Hierarchical) were fitted to the behavioral data from the metacontrast masking paradigm. We tested multiple versions of each category of model (see “Materials and Methods” section for details). Shown here are the best-fitting models from each category, selected according to formal model comparison techniques (Fig. 6). The Hierarchical model performed best at capturing the dissociation between task performance and reported levels of stimulus visibility. This dissociation is made readily apparent by plotting visibility reports against task performance, as depicted in the bottom row of figures; the relationship is not monotonic, but exhibits a sharp spike at around 80–85% correct, reflecting that short SOAs had lower visibility than long SOAs in spite of having similar task performanceCC BY-NC · PubMed Central
Figure 5. Model fits for type 2 data. In addition to the distinctive dissociation between task performance and visibility (Fig. 4), the behavioral data also included a set of type 2 data that provided a challenge for model fitting. By “type 2 data,” we refer to the probability of giving different levels of visibility ratings conditional upon task performance. (Top panel) Type 2 hit rate (HR; probability of high visibility for correct responses) and type 2 false alarm rate (FAR; probability of high visibility for incorrect responses) as a function of SOA. Note that for ease of visualization, here we plot only a single (type 2 HR, type 2 FAR) pair based on a median split of the visibility rating scale, although in principle a four-point rating scale yields three such pairs, since there are three ways to collapse the four-point scale into a binary distinction between “high” and “low” visibility. (Bottom panel) meta-d’ as a function of d’, where each data point corresponds to one SOA. SDT models predict that meta-d’ = d’, i.e. that increases in task performance manifest as increases in the discriminability of correct and incorrect trials by subjective ratings of confidence or awareness. This improved discriminability of task performance manifests as a divergence of type 2 hit rate and type 2 false alarm rate. Although meta-d’ was lower than d’ at all SOA in this dataset, the Single Channel model could not reproduce this pattern as it is constrained such that it can only yield values of meta-d’ equal to d’ (Maniscalco and Lau, 2012; Maniscalco and Lau, 2014). The best performing Dual Channel model is not rigidly constrained in this way, but still produced meta-d’ values close to d’. In contrast, the Hierarchical model produced a meta-d’ vs d’ curve that lies well below the line of unity, providing a closer fit to the data. Note that meta-d’ was computed using all three available (type 2 HR, type 2 FAR) pairs for each SOA, i.e. it was not computed based on a median split of visibility
Figure 5. Model fits for type 2 data. In addition to the distinctive dissociation between task performance and visibility (Fig. 4), the behavioral data also included a set of type 2 data that provided a challenge for model fitting. By “type 2 data,” we refer to the probability of giving different levels of visibility ratings conditional upon task performance. (Top panel) Type 2 hit rate (HR; probability of high visibility for correct responses) and type 2 false alarm rate (FAR; probability of high visibility for incorrect responses) as a function of SOA. Note that for ease of visualization, here we plot only a single (type 2 HR, type 2 FAR) pair based on a median split of the visibility rating scale, although in principle a four-point rating scale yields three such pairs, since there are three ways to collapse the four-point scale into a binary distinction between “high” and “low” visibility. (Bottom panel) meta-d’ as a function of d’, where each data point corresponds to one SOA. SDT models predict that meta-d’ = d’, i.e. that increases in task performance manifest as increases in the discriminability of correct and incorrect trials by subjective ratings of confidence or awareness. This improved discriminability of task performance manifests as a divergence of type 2 hit rate and type 2 false alarm rate. Although meta-d’ was lower than d’ at all SOA in this dataset, the Single Channel model could not reproduce this pattern as it is constrained such that it can only yield values of meta-d’ equal to d’ (Maniscalco and Lau, 2012; Maniscalco and Lau, 2014). The best performing Dual Channel model is not rigidly constrained in this way, but still produced meta-d’ values close to d’. In contrast, the Hierarchical model produced a meta-d’ vs d’ curve that lies well below the line of unity, providing a closer fit to the data. Note that meta-d’ was computed using all three available (type 2 HR, type 2 FAR) pairs for each SOA, i.e. it was not computed based on a median split of visibilityCC BY-NC · PubMed Central
Figure 6. Model selection results. Formal model comparison was conducted using a finite sample size correction of the Akaike Information Criterion (AICc), which rewards models for closely fitting observed data while punishing models for the degree of complexity (i.e. number of free parameters; for list of free parameters for all models please see “Materials and Methods” section). For ease of interpretation, we display a transformation of AICc values into Akaike weights, which quantify the information theoretic evidence in favor of each model such that the weights sum to 1 (Burnham and Anderson, 2002). (Top panel) Model selection on all 26 models. Black bars plot the across-subject average of the Akaike weights that were maximal in each model class for each subject. White bars plot the probability that the maximum Akaike weight across all 26 models belonged to a given model class. For both measures, the value for the Hierarchical model class was roughly three times higher than that of the Dual Channel model class and roughly four times higher than that of the Single Channel model class. (Bottom panel) Similar results were found when restricting the analysis to the best models in each model class. Best models were defined by computing the average Akaike weight for all 26 models, and then selecting the models in each model class with the maximum average Akaike weight. Full model comparison results can be found in Table 1
Figure 6. Model selection results. Formal model comparison was conducted using a finite sample size correction of the Akaike Information Criterion (AICc), which rewards models for closely fitting observed data while punishing models for the degree of complexity (i.e. number of free parameters; for list of free parameters for all models please see “Materials and Methods” section). For ease of interpretation, we display a transformation of AICc values into Akaike weights, which quantify the information theoretic evidence in favor of each model such that the weights sum to 1 (Burnham and Anderson, 2002). (Top panel) Model selection on all 26 models. Black bars plot the across-subject average of the Akaike weights that were maximal in each model class for each subject. White bars plot the probability that the maximum Akaike weight across all 26 models belonged to a given model class. For both measures, the value for the Hierarchical model class was roughly three times higher than that of the Dual Channel model class and roughly four times higher than that of the Single Channel model class. (Bottom panel) Similar results were found when restricting the analysis to the best models in each model class. Best models were defined by computing the average Akaike weight for all 26 models, and then selecting the models in each model class with the maximum average Akaike weight. Full model comparison results can be found in Table 1CC BY-NC · PubMed Central
Figure 7. Parameter values from model fits. Parameters for the best fitting models from each model class—“Single Channel CV,” “Weighted Dual Channel,” and “Constant Noise + Decay”—are plotted. In addition to the depicted parameters, each model fit also specified values for seven decision criteria (data not shown). For descriptions of model structure and parameters, see “Materials and Methods” section
Figure 7. Parameter values from model fits. Parameters for the best fitting models from each model class—“Single Channel CV,” “Weighted Dual Channel,” and “Constant Noise + Decay”—are plotted. In addition to the depicted parameters, each model fit also specified values for seven decision criteria (data not shown). For descriptions of model structure and parameters, see “Materials and Methods” sectionCC BY-NC · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 152

Metadata

Authors
Maniscalco, B., & Lau, H.
Affiliations
  • National Institutes of Health (US)
  • University of California, Los Angeles (US)
Corresponding author
Maniscalco, Brian
Venue
Neuroscience of Consciousness
Published
2016-01-01
Volume
2016
Issue
1
ISSN
2057-2107 (ISSN-L)
OpenAlex
W2335867627
Cited by
133
References
69
Citations by year
2026: 8 · 2025: 7 · 2024: 15 · 2023: 17 · 2022: 13 · 2021: 15 · 2020: 18 · 2019: 10 · 2018: 10 · 2017: 15 · 2016: 4
FWCI
8.27
Type
journal article
Language
en
Open access
Locations
  • Neuroscience of Consciousness (publishedVersion, cc-by-nc)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
Funders
National Institutes of Health
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
  • Social Sciences › Arts and Humanities › Philosophy › Philosophy and Theoretical Science
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
Keywords
visual awareness, metacontrast masking, stimulus identification, perceptual consciousness, dual-channel model, hierarchical models, single-channel model, model comparison, computational models of perception

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.