The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Decoding the Semantic Content of Natural Movies from Human Brain Activity
Open accessAbstract
One crucial test for any quantitative model of the brain is to show that the model can be used to accurately decode information from evoked brain activity. Several recent neuroimaging studies have decoded the structure or semantic content of static visual images from human brain activity. Here we present a decoding algorithm that makes it possible to decode detailed information about the object and action categories present in natural movies from human brain activity signals measured by functional MRI. Decoding is accomplished using a hierarchical logistic regression (HLR) model that is based on labels that were manually assigned from the WordNet semantic taxonomy. This model makes it possible to simultaneously decode information about both specific and general categories, while respecting the relationships between them. Our results show that we can decode the presence of many object and action categories from averaged blood-oxygen level-dependent (BOLD) responses with a high degree of accuracy (area under the ROC curve > 0.9). Furthermore, we used this framework to test whether semantic relationships defined in the WordNet taxonomy are represented the same way in the human brain. This analysis showed that hierarchical relationships between general categories and atypical examples, such as organism and plant, did not seem to be reflected in representations measured by BOLD fMRI.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 2×
“Our decoder used a HLR model based on the graphical structure of WordNet, a semantic taxonomy that was manually constructed by a team of linguists (Miller, 1995).”
MethodThe fusiform face area: a module in human extrastriate cortex specialized for face perception
Kanwisher, N., McDermott, J., & Chun, M. M. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-11-04302.1997 · Cited by 7,738
Basic objects in natural categories
Rosch, E., Mervis, C. B., Gray, W. D., Johnson, D. M., & Boyes-Braem, P. (1976) · Cognitive Psychology · doi:10.1016/0010-0285(76)90013-x · Cited by 5,643 · cited 2×
“One solution to this issue would be to decode only one label in the hierarchy, such as the basic-level category (Rosch et al., 1976), which in this example would likely be car .”
BackgroundMethodA cortical area selective for visual processing of the human body
Downing, P. E., Jiang, Y., Shuman, M., & Kanwisher, N. (2001) · Science · doi:10.1126/science.1063414 · Cited by 2,043
Predicting Human Brain Activity Associated with the Meanings of Nouns
Mitchell, T. M., Shinkareva, S. V., Carlson, A., Chang, K.-M., Malave, V. L., Mason, R. A., & Just, M. A. (2008) · Science · doi:10.1126/science.1152876 · Cited by 1,180
Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies
Nishimoto, S., Vu, A. T., Naselaris, T., Benjamini, Y., Yu, B., & Gallant, J. L. (2011) · Current Biology · doi:10.1016/j.cub.2011.08.031 · Cited by 933
Self-citation
Cited by
Broderick, M. P., Anderson, A. J., Di Liberto, G. M., Crosse, M. J., & Lalor, E. C. (2018) · Current Biology · doi:10.1016/j.cub.2018.01.080 · Cited by 421
Should a Few Null Findings Falsify Prefrontal Theories of Conscious Perception?
Odegaard, B., Knight, R. T., & Lau, H. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.3217-16.2017 · Cited by 316
Are we really unconscious in “unconscious” states? Common assumptions revisited
Sevenius Nilsen, A., Juel, B. E., Thürer, B., Aamodt, A., & Storm, J. F. (2022) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2022.987051 · Cited by 49
Brain Decoding from Functional MRI Using Long Short-Term Memory Recurrent Neural Networks
Li, H., & Fan, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00931-1_37 · Cited by 45
Levin, M. (2025) · BioEssays · doi:10.1002/bies.202400196 · Cited by 34
Metadata
- Authors
- Huth, A. G., Lee, T., Nishimoto, S., Bilenko, N. Y., Vu, A. T., & Gallant, J. L.
- Venue
- Frontiers in Systems Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2016-10-07
- Volume
- 10
- Pages
- 81-81
- ISSN
- 1662-5137 (ISSN-L)
- OpenAlex
- W2528108497
- Cited by
- 163
- References
- 6
- FWCI
- 7.84
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.