Sociality and interaction envelope organize visual action representations
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Humans observe a wide range of actions in their surroundings. How is the visual cortex organized to process this diverse input? Using functional neuroimaging, we measured brain responses while participants viewed short videos of everyday actions, then probed the structure in these responses using voxel-wise encoding modeling. Responses are well fit by feature spaces that capture the body parts involved in an action and the action’s targets (i.e. whether the action was directed at an object, another person, the actor, and space). Clustering analyses reveal five large-scale networks that summarize the voxel tuning: one related to social aspects of an action, and four related to the scale of the interaction envelope, ranging from fine-scale manipulations directed at objects, to large-scale whole-body movements directed at distant locations. We propose that these networks reveal the major representational joints in how actions are processed by visual regions of the brain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965 · cited 1×
“Method Details Stimulus Set 120 2.5-second videos of 60 everyday actions were collected from YouTube, Vine, the Human Movement Database (Kuehne et al., 2011), and the University of Central Florida’s Action Recognition Data Set (Soomro, Zamir, & Shah, 2012).”
MethodInfluentialA cortical area selective for visual processing of the human body
Downing, P. E., Jiang, Y., Shuman, M., & Kanwisher, N. (2001) · Science · doi:10.1126/science.1063414 · Cited by 2,043
ALE meta-analysis of action observation and imitation in the human brain
Caspers, S., Zilles, K., Laird, A. R., & Eickhoff, S. B. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.112 · Cited by 1,386
Predicting Human Brain Activity Associated with the Meanings of Nouns
Mitchell, T. M., Shinkareva, S. V., Carlson, A., Chang, K.-M., Malave, V. L., Mason, R. A., & Just, M. A. (2008) · Science · doi:10.1126/science.1152876 · Cited by 1,180 · cited 3×
“The present work follows this latter approach, leveraging an encoding analytical framework to characterize action responses in the visual system (Mitchell et al., 2008; Huth et al., 2012; Lescroart et al., 2015).”
MethodShared memories reveal shared structure in neural activity across individuals
Chen, J., Leong, Y. C., Honey, C. J., Yong, C. H., Norman, K. A., & Hasson, U. (2017) · Nature Neuroscience · doi:10.1038/nn.4450 · Cited by 645
A distributed left hemisphere network active during planning of everyday tool use skills
Johnson-Frey, S. H., Newman-Norlund, R., & Grafton, S. T. (2005) · Cerebral Cortex · doi:10.1093/cercor/bhh169 · Cited by 558
Evidence for a distributed hierarchy of action representation in the brain
Grafton, S. T., & de C. Hamilton, A. F. (2007) · Human Movement Science · doi:10.1016/j.humov.2007.05.009 · Cited by 503
A region of right posterior superior temporal sulcus responds to observed intentional actions
Saxe, R., Xiao, D.-K., Kovacs, G., Perrett, D. I., & Kanwisher, N. (2004) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2004.04.015 · Cited by 446 · cited 2×
“It is near STS regions typically engaged by social processing (Saxe et al., 2004; Deen et al., 2015; Isik et al., 2017) while also extending inferiorly into ventral OTC along the fusiform gyrus.”
BackgroundResultFunctional organization of social perception and cognition in the superior temporal sulcus
Deen, B., Koldewyn, K., Kanwisher, N., & Saxe, R. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhv111 · Cited by 443
Swettenham, J., Baron-Cohen, S., Charman, T., Cox, A., Baird, G., Drew, A., Rees, L., & Wheelwright, S. (1998) · Journal of Child Psychology and Psychiatry · doi:10.1017/s0021963098002595 · Cited by 398
Cited by
Bonner, M. F., & Epstein, R. A. (2021) · Nature Communications · doi:10.1038/s41467-021-24368-2 · Cited by 96
The Language Network Is Recruited but Not Required for Nonverbal Event Semantics
Ivanova, A. A., Mineroff, Z., Zimmerer, V., Kanwisher, N., Varley, R., & Fedorenko, E. (2021) · Neurobiology of Language · doi:10.1162/nol_a_00030 · Cited by 65
“…specialized neural mechanisms, distinct from the language network, have been associated with visual understanding of actions (Fang et al., 2016; Häberling et al., 2016; Tarhan & Konkle, 2020) and interactions between animate and/or inanimate entities (Fischer et al., 2016; Walbrin et al., 2018).”
BackgroundUniversality of representation in biological and artificial neural networks
Hosseini, E., Casto, C., Zaslavsky, N., Conwell, C., Richardson, M., & Fedorenko, E. (2024) · doi:10.1101/2024.12.26.629294 · Cited by 10
High-dimensional structure underlying individual differences in naturalistic visual experience
Han, C., & Bonner, M. F. (2026) · Current Biology · doi:10.1016/j.cub.2025.12.039 · Cited by 5
Metadata
- Authors
- Tarhan, L., & Konkle, T.
- Venue
- Nature Communications
- Published
- 2020-06-12
- Volume
- 11
- Issue
- 1
- Pages
- 3002-3002
- ISSN
- 2041-1723 (ISSN-L)
- OpenAlex
- W3034731391
- Cited by
- 72
- References
- 14
- FWCI
- 11.10
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Sociality and interaction envelope organize visual action representations. https://app.cochat.ai/papers/10.1038/s41467-020-16846-w
- The paper
- Tarhan, L., & Konkle, T. (2020). Sociality and interaction envelope organize visual action representations. Nature Communications. https://doi.org/10.1038/s41467-020-16846-w
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.