Interaction between Scene and Object Processing Revealed by Human fMRI and MEG Decoding
Open accessAbstract
Scenes strongly facilitate object recognition, such as when we make out the shape of a distant boat on the water. Yet, although known to interact in perception, neuroimaging research has primarily provided evidence for separate scene- and object-selective cortical pathways. This raises the question of how these pathways interact to support context-based perception. Here we used a novel approach in human fMRI and MEG studies to reveal supra-additive scene-object interactions. Participants (men and women) viewed degraded objects that were hard to recognize when presented in isolation but easy to recognize within their original scene context, in which no other associated objects were present. fMRI decoding showed that the multivariate representation of the objects9 category (animate/inanimate) in object-selective cortex was strongly enhanced by the presence of scene context, even though the scenes alone did not evoke category-selective response patterns. This effect in object-selective cortex was correlated with concurrent activity in scene-selective regions. MEG decoding results revealed that scene-based facilitation of object processing peaked at 320 ms after stimulus onset, 100 ms later than peak decoding of intact objects. Together, results suggest that expectations derived from scene information, processed in scene-selective cortex, feed back to shape object representations in visual cortex. These findings characterize, in space and time, functional interactions between scene- and object-processing pathways. SIGNIFICANCE STATEMENT Although scenes and objects are known to contextually interact in visual perception, the study of high-level vision has mostly focused on the dissociation between their selective neural pathways. The current findings are the first to reveal direct facilitation of object recognition and neural representation by scene background, even in the absence of contextually associated objects. Using a multivariate approach to both fMRI and MEG, we characterize the functional neuroanatomy and neural dynamics of such scene-based object facilitation. Finally, the correlation of this effect with scene-selective activity suggests that, although functionally distinct, scene and object processing pathways do interact at a perceptual level to fill in for insufficient visual detail.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Object representations are modulated by scene context.
“Indeed, there is evidence that the neural representation of simple stimulus features is influenced by higher‐level object representations 115 , 129 and object representations are modulated by scene context.”
From Evaluating the neurophysiological evidence for predictive processing as a model of perception (2020) · cites this paper as [197]
Neural representations of objects are enhanced by the presence of scene context, and this enhancement is correlated with the engagement of scene-selective cortical regions.
“For example, neural representations of objects are enhanced by the presence of scene context, and this enhancement is correlated with the engagement of sceneselective cortical regions [32]”
From How Do Expectations Shape Perception? (2018) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162 · see citing works
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · see citing works
SMITH, S., & NICHOLS, T. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.03.061 · Cited by 5,250 · see citing works
The Organization of Behavior; A Neuropsychological Theory.
Attneave, F., B., M., & Hebb, D. O. (1950) · The American Journal of Psychology · doi:10.2307/1418888 · Cited by 5,146 · see citing works
The Organization of Behavior: A Neuropsychological Theory
(1950) · JAMA · doi:10.1001/jama.1950.02910470083028 · Cited by 4,271 · see citing works
Distributed and overlapping representations of faces and objects in ventral temporal cortex
Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001) · Science · doi:10.1126/science.1063736 · Cited by 3,940 · see citing works
A cortical representation of the local visual environment
Epstein, R. A., & Kanwisher, N. (1998) · Nature · doi:10.1038/33402 · Cited by 2,960 · see citing works
Information-based functional brain mapping
Kriegeskorte, N., Goebel, R., & Bandettini, P. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0600244103 · Cited by 2,113 · see citing works
Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex.
Malach, R., Reppas, J. B., Benson, R. R., Kwong, K. K., Jiang, H., Kennedy, W. A., Ledden, P. J., Brady, T. J., Rosen, B. R., & Tootell, R. B. H. (1995) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.92.18.8135 · Cited by 1,761 · see citing works
Bar, M. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1476 · Cited by 1,613 · see citing works
Multisensory integration: current issues from the perspective of the single neuron
Stein, B. E., & Stanford, T. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2331 · Cited by 1,598 · see citing works
Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey
Kriegeskorte, N., Mur, M., Ruff, D. A., Kiani, R., Bodurka, J., Esteky, H., Tanaka, K., & Bandettini, P. A. (2008) · Neuron · doi:10.1016/j.neuron.2008.10.043 · Cited by 1,334 · see citing works
The role of context in object recognition
Oliva, A., & Torralba, A. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.09.009 · Cited by 1,063 · see citing works
Scene perception: Detecting and judging objects undergoing relational violations
Biederman, I., Mezzanotte, R. J., & Rabinowitz, J. C. (1982) · Cognitive Psychology · doi:10.1016/0010-0285(82)90007-x · Cited by 1,059 · see citing works
Predictive coding: a fresh view of inhibition in the retina
Srinivasan, M. V., Laughlin, S. B., & Dubs, A. (1982) · Proceedings of the Royal Society of London. Series B. Biological Sciences · doi:10.1098/rspb.1982.0085 · Cited by 1,042 · see citing works
The role of the parahippocampal cortex in cognition
Aminoff, E., Kveraga, K., & Bar, M. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.06.009 · Cited by 855 · see citing works
Expectation in perceptual decision making: neural and computational mechanisms
Summerfield, C., & de Lange, F. P. (2014) · Nature Reviews Neuroscience · doi:10.1038/nrn3838 · Cited by 832 · see citing works
Resolving human object recognition in space and time
Cichy, R. M., Pantazis, D., & Oliva, A. (2014) · Nature Neuroscience · doi:10.1038/nn.3635 · Cited by 832 · see citing works
CoSMoMVPA: Multi-Modal Multivariate Pattern Analysis of Neuroimaging Data in Matlab/GNU Octave
Oosterhof, N. N., Connolly, A. C., & Haxby, J. V. (2016) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2016.00027 · Cited by 660 · see citing works
Cortical analysis of visual context.
Bar, M., & Aminoff, E. (2003) · Neuron · doi:10.1016/s0896-6273(03)00167-3 · Cited by 630 · see citing works
Scene Consistency in Object and Background Perception
Davenport, J. L., & Potter, M. C. (2004) · Psychological Science · doi:10.1111/j.0956-7976.2004.00719.x · Cited by 567 · see citing works
Huang, Y., & Rao, R. P. N. (2011) · WIREs Cognitive Science · doi:10.1002/wcs.142 · Cited by 470 · see citing works
Cortical Mechanisms Specific to Explicit Visual Object Recognition
Bar, M., Tootell, R. B. H., Schacter, D. L., Greve, D. N., Fischl, B. R., Mendola, J. D., Rosen, B. R., & Dale, A. M. (2001) · Neuron · doi:10.1016/s0896-6273(01)00224-0 · Cited by 441 · see citing works
Eliaš, P. (1955) · IEEE Transactions on Information Theory · doi:10.1109/tit.1955.1055126 · Cited by 436 · see citing works
Representational dynamics of object vision: The first 1000 ms
Carlson, T. A., Tovar, D. A., Alink, A., & Kriegeskorte, N. (2013) · Journal of Vision · doi:10.1167/13.10.1 · Cited by 413 · see citing works
Stelzer, J., Chen, Y., & Turner, R. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2012.09.063 · Cited by 372 · see citing works
The Occipital Place Area Is Causally and Selectively Involved in Scene Perception
Dilks, D. D., Julian, J. B., Paunov, A. M., & Kanwisher, N. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.4081-12.2013 · Cited by 332 · see citing works
An electrophysiological study of scene effects on object identification
Ganis, G., & Kutas, M. (2003) · Cognitive Brain Research · doi:10.1016/s0926-6410(02)00244-6 · Cited by 264 · see citing works
Processing scene context: Fast categorization and object interference
Joubert, O. R., Rousselet, G. A., Fize, D., & Fabre‐Thorpe, M. (2007) · Vision Research · doi:10.1016/j.visres.2007.09.013 · Cited by 263 · see citing works
Timecourse of neural signatures of object recognition
Johnson, J. S., & Olshausen, B. A. (2003) · Journal of Vision · doi:10.1167/3.7.4 · Cited by 235 · see citing works
Differential Electrophysiological Signatures of Semantic and Syntactic Scene Processing
Võ, M. L. ‐., & Wolfe, J. M. (2013) · Psychological Science · doi:10.1177/0956797613476955 · Cited by 228 · see citing works
Spatial Context in Recognition
Bar, M., & Ullman, S. (1996) · Perception · doi:10.1068/p250343 · Cited by 227 · see citing works
Contextually Evoked Object-Specific Responses in Human Visual Cortex
Cox, D., Meyers, E. M., & Sinha, P. (2004) · Science · doi:10.1126/science.1093110 · Cited by 195 · see citing works
Constructing scenes from objects in human occipitotemporal cortex
MacEvoy, S. P., & Epstein, R. A. (2011) · Nature Neuroscience · doi:10.1038/nn.2903 · Cited by 180 · see citing works
Proklova, D., KAISER, D. H., & Peelen, M. V. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00924 · Cited by 179 · see citing works
Self-citationCichy, R. M., Khosla, A., Pantazis, D., & Oliva, A. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.03.063 · Cited by 179 · see citing works
Deconstructing Visual Scenes in Cortex: Gradients of Object and Spatial Layout Information
Harel, A., Kravitz, D. J., & Baker, C. I. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhs091 · Cited by 152 · see citing works
Identification of objects in scenes: The role of scene background in object naming.
Boyce, S. J., & Pollatsek, A. (1992) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037//0278-7393.18.3.531 · Cited by 139 · see citing works
Early onset of neural synchronization in the contextual associations network
Kveraga, K., Ghuman, A. S., Kassam, K., Aminoff, E., Hämäläinen, M., Chaumon, M., & Bar, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1013760108 · Cited by 114 · see citing works
The Neural Dynamics of Attentional Selection in Natural Scenes
KAISER, D. H., Oosterhof, N. N., & Peelen, M. V. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.1385-16.2016 · Cited by 100 · see citing works
Self-citationHollingworth, A., & Henderson, J. M. (1999) · Acta Psychologica · doi:10.1016/s0001-6918(98)00053-5 · Cited by 84 · see citing works
Identification of objects in scenes: The role of scene background in object naming.
Boyce, S. J., & Pollatsek, A. (1992) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.18.3.531 · Cited by 84 · see citing works
The Body Inversion Effect Is Mediated by Face-Selective, Not Body-Selective, Mechanisms
Brandman, T., & Yovel, G. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.0911-10.2010 · Cited by 76 · see citing works
Self-citationSynchronous contextual irregularities affect early scene processing: Replication and extension
Mudrik, L., Shalgi, S., Lamy, D., & Deouell, L. Y. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.02.020 · Cited by 75 · see citing works
TMS to the Lateral Occipital Cortex Disrupts Object Processing but Facilitates Scene Processing
Mullin, C. R., & Steeves, J. K. E. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00095 · Cited by 71 · see citing works
The influence of scene context on object recognition is independent of attentional focus
Munneke, J., Brentari, V., & Peelen, M. V. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00552 · Cited by 61 · see citing works
Self-citationDemiral, Ş. B., Malcolm, G. L., & Henderson, J. M. (2012) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2012.02.011 · Cited by 54 · see citing works
Incongruent object/context relationships in visual scenes: Where are they processed in the brain?
Rémy, F., Vayssière, N., Pins, D., Boucart, M., & Fabre‐Thorpe, M. (2013) · Brain and Cognition · doi:10.1016/j.bandc.2013.10.008 · Cited by 48 · see citing works
Vertical and horizontal references determined by linear perspective and optic flow information
Wu, J., He, Z., & Leng, T. (2010) · Journal of Vision · doi:10.1167/3.9.499 · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
How Do Expectations Shape Perception?
de Lange, F. P., Heilbron, M., & Kok, P. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.06.002 · Cited by 975 · see citing works
Evaluating the neurophysiological evidence for predictive processing as a model of perception
Walsh, K. S., McGovern, D. P., Clark, A., & O'Connell, R. G. (2020) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.14321 · Cited by 333 · see citing works
Forms of prediction in the nervous system
Teufel, C., & Fletcher, P. C. (2020) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-0275-5 · Cited by 155 · see citing works
Bonner, M. F., & Epstein, R. A. (2021) · Nature Communications · doi:10.1038/s41467-021-24368-2 · Cited by 96 · see citing works
Linking cortical circuit models to human cognition with laminar fMRI
Yang, J., Huber, L., Yu, Y., & Bandettini, P. A. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.07.005 · Cited by 52 · see citing works
Word contexts enhance the neural representation of individual letters in early visual cortex
Heilbron, M., Richter, D., Ekman, M., Hagoort, P., & de Lange, F. P. (2020) · Nature Communications · doi:10.1038/s41467-019-13996-4 · Cited by 48 · see citing works
Qianchen, L., Gallagher, R. M., & Tsuchiya, N. (2022) · Royal Society Open Science · doi:10.1098/rsos.210394 · Cited by 21 · see citing works
- Authors
- Brandman, T., & Peelen, M. V.
- Affiliations
- University of Trento (IT)
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2017-07-07
- Volume
- 37
- Issue
- 32
- Pages
- 7700-7710
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- PMC
- PMC6596648
- OpenAlex
- W2732001149
- Cited by
- 168
- References
- 50
- Citations by year
- 2026: 18 · 2025: 31 · 2024: 16 · 2023: 21 · 2022: 15 · 2021: 18 · 2020: 22 · 2019: 16 · 2018: 10 · 2017: 1
- FWCI
- 7.69
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Neuroscience (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Radboud Repository (Radboud University) (submittedVersion)
- Funders
- European Commission (HORIZON2020, 659778)
- Provincia Autonoma di Trento
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- scene-selective cortex, object-selective cortex, object recognition, scene context, neural dynamics, multivariate decoding, visual cortex
- MeSH terms
- Adult, Female, Humans, Magnetic Resonance Imaging, Male, Middle Aged, Pattern Recognition, Visual, Photic Stimulation, Visual Cortex, Visual Perception, Magnetoencephalography, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
