Resolving the neural dynamics of visual and auditory scene processing in the human brain: a methodological approach
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
In natural environments, visual and auditory stimulation elicit responses across a large set of brain regions in a fraction of a second, yielding representations of the multimodal scene and its properties. The rapid and complex neural dynamics underlying visual and auditory information processing pose major challenges to human cognitive neuroscience. Brain signals measured non-invasively are inherently noisy, the format of neural representations is unknown, and transformations between representations are complex and often nonlinear. Further, no single non-invasive brain measurement technique provides a spatio-temporally integrated view. In this opinion piece, we argue that progress can be made by a concerted effort based on three pillars of recent methodological development: (i) sensitive analysis techniques such as decoding and cross-classification, (ii) complex computational modelling using models such as deep neural networks, and (iii) integration across imaging methods (magnetoencephalography/electroencephalography, functional magnetic resonance imaging) and models, e.g. using representational similarity analysis. We showcase two recent efforts that have been undertaken in this spirit and provide novel results about visual and auditory scene analysis. Finally, we discuss the limits of this perspective and sketch a concrete roadmap for future research. This article is part of the themed issue ‘Auditory and visual scene analysis’.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
What we can do and what we cannot do with fMRI
Logothetis, N. K. (2008) · Nature · doi:10.1038/nature06976 · Cited by 3,612 · cited 2×
“fMRI is sensitive to blood oxygenation levels [4], which correlate with local neuronal activity.”
BackgroundRepresentational similarity analysis-connecting the branches of systems neuroscience
Kriegeskorte, N. (2008) · Frontiers in Systems Neuroscience · doi:10.3389/neuro.06.004.2008 · Cited by 3,355
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
Circular analysis in systems neuroscience: the dangers of double dipping
Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F., & Baker, C. I. (2009) · Nature Neuroscience · doi:10.1038/nn.2303 · Cited by 2,696
Beyond mind-reading: multi-voxel pattern analysis of fMRI data
Norman, K. A., Polyn, S. M., Detre, G. J., & Haxby, J. V. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.07.005 · Cited by 2,271
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
A new neural framework for visuospatial processing
Kravitz, D. J., Saleem, K. S., Baker, C. I., & Mishkin, M. (2011) · Nature Reviews Neuroscience · doi:10.1038/nrn3008 · Cited by 1,325
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
Guclu, U., & van Gerven, M. A. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5023-14.2015 · Cited by 961
Cited by
To integrate or not to integrate: Temporal dynamics of hierarchical Bayesian causal inference
Aller, M., & Noppeney, U. (2019) · PLOS Biology · doi:10.1371/journal.pbio.3000210 · Cited by 98
A roadmap for the study of conscious audition and its neural basis
Dykstra, A. R., Cariani, P. A., & Gutschalk, A. (2017) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2016.0103 · Cited by 61
The Embodied Teaching of Spatial Terms: Gestures Mapped to Morphemes Improve Learning
Janzen Ulbricht, N. (2020) · Frontiers in Education · doi:10.3389/feduc.2020.00109 · Cited by 13
Metadata
- Authors
- Cichy, R. M., & Teng, S.
- Venue
- Philosophical Transactions of the Royal Society B: Biological Sciences
- Publisher
- The Royal Society
- Published
- 2017-02-19
- Volume
- 372
- Issue
- 1714
- Pages
- 20160108
- ISSN
- 0962-8436 (ISSN-L)
- 1471-2970
- OpenAlex
- W2566385122
- Cited by
- 38
- References
- 15
- FWCI
- 6.79
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.