Movies and narratives as naturalistic stimuli in neuroimaging
Open accessThe DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
Abstract
Using movies and narratives as naturalistic stimuli in human neuroimaging studies has yielded significant advances in understanding of cognitive and emotional functions. The relevant literature was reviewed, with emphasis on how the use of naturalistic stimuli has helped advance scientific understanding of human memory, attention, language, emotions, and social cognition in ways that would have been difficult otherwise. These advances include discovering a cortical hierarchy of temporal receptive windows, which supports processing of dynamic information that accumulates over several time scales, such as immediate reactions vs. slowly emerging patterns in social interactions. Naturalistic stimuli have also helped elucidate how the hippocampus supports segmentation and memorization of events in day-to-day life and have afforded insights into attentional brain mechanisms underlying our ability to adopt specific perspectives during natural viewing. Further, neuroimaging studies with naturalistic stimuli have revealed the role of the default-mode network in narrative-processing and in social cognition. Finally, by robustly eliciting genuine emotions, these stimuli have helped elucidate the brain basis of both basic and social emotions apparently manifested as highly overlapping yet distinguishable patterns of brain activity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Control of goal-directed and stimulus-driven attention in the brain
Corbetta, M., & Shulman, G. L. (2002) · Nature Reviews Neuroscience · doi:10.1038/nrn755 · Cited by 12,881
The cortical organization of speech processing
Hickok, G., & Poeppel, D. (2007) · Nature Reviews Neuroscience · doi:10.1038/nrn2113 · Cited by 5,495
Perception of the speech code.
Liberman, A. M., Cooper, F. S., Shankweiler, D. P., & Studdert-Kennedy, M. (1967) · Psychological Review · doi:10.1037/h0020279 · Cited by 3,722
Intersubject synchronization of cortical activity during natural vision
Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., & Malach, R. (2004) · Science · doi:10.1126/science.1089506 · Cited by 1,839 · cited 1×
“The ISC is very well suited for analysis of fMRI data since it is a model-free approach, where one does not need to build predictors of specific events in the rich and complex movie stimuli as in case of general-linear model (GLM) based analyses ( Hasson et al., 2004 ).”
Natural speech reveals the semantic maps that tile human cerebral cortex
Huth, A. G., de Heer, W. A., Griffiths, T. L., Theunissen, F. E., & Gallant, J. L. (2016) · Nature · doi:10.1038/nature17637 · Cited by 1,504
Shamay-Tsoory, S. G. (2011) · The Neuroscientist · doi:10.1177/1073858410379268 · Cited by 1,059
Discovering Event Structure in Continuous Narrative Perception and Memory
Baldassano, C., Chen, J., Zadbood, A., Pillow, J. W., Hasson, U., & Norman, K. A. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.041 · Cited by 952
A hierarchy of temporal receptive windows in human cortex
Hasson, U., Yang, E., Vallines, I., Heeger, D. J., & Rubin, N. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.5487-07.2008 · Cited by 874
Topographic mapping of a hierarchy of temporal receptive windows using a narrated story
Lerner, Y., Honey, C. J., Silbert, L. J., & Hasson, U. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3684-10.2011 · Cited by 834 · cited 1×
“A subsequent study, in which reordering of audio narratives utilized several segment lengths, showed an analogous hierarchy of TRWs ( Lerner et al., 2011 ).”
Hierarchical process memory: memory as an integral component of information processing
Hasson, U., Chen, J., & Honey, C. J. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.04.006 · Cited by 749
Neuroanatomical correlates of externally and internally generated human emotion
(1997) · American Journal of Psychiatry · doi:10.1176/ajp.154.7.918 · Cited by 711 · cited 1×
“Today, several decades later, movies and narratives have seen use as naturalistic stimuli in a wealth of neuroimaging studies (Aftanas et al., 1998; Cahill et al., 1996; Decety and Chaminade, 2003; Dimpfel et al., 2003; Dougherty et al., 1999; Gilbert et al., 1989; Jones and Fox, 1992; Karama et al., 2002; Krause et a…”
BackgroundShared memories reveal shared structure in neural activity across individuals
Chen, J., Leong, Y. C., Honey, C. J., Yong, C. H., Norman, K. A., & Hasson, U. (2017) · Nature Neuroscience · doi:10.1038/nn.4450 · Cited by 645
Dynamic reconfiguration of the default mode network during narrative comprehension
Simony, E., Honey, C. J., Chen, J., Lositsky, O., Yeshurun, Y., Wiesel, A., & Hasson, U. (2016) · Nature Communications · doi:10.1038/ncomms12141 · Cited by 588 · cited 1×
“Studies have shown that comprehension of narrative-level information that accumulates on larger timescales depends on DMN structures (AbdulSabur et al., 2014; Babajani-Feremi, 2017; Gordon et al., 2017; Nguyen et al., 2019; Regev et al., 2013; Simony et al., 2016; Tikka et al., 2018; Wilson et al., 2008; Xu et al., 20…”
BackgroundHuman brain activity time-locked to perceptual event boundaries
Zacks, J. M., Braver, T. S., Sheridan, M. A., Donaldson, D. I., Snyder, A. Z., Ollinger, J. M., Buckner, R. L., & Raichle, M. E. (2001) · Nature Neuroscience · doi:10.1038/88486 · Cited by 572
Emotions promote social interaction by synchronizing brain activity across individuals
Nummenmaa, L., Glerean, E., Viinikainen, M., Jääskeläinen, I. P., Hari, R., & Sams, M. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1206095109 · Cited by 538
Self-citationInternally generated reactivation of single neurons in human hippocampus during free recall
Gelbard-Sagiv, H., Mukamel, R., Harel, M., Malach, R., & Fried, I. (2008) · Science · doi:10.1126/science.1164685 · Cited by 533
Slow cortical dynamics and the accumulation of information over long timescales
Honey, C. J., Thesen, T., Donner, T. H., Silbert, L. J., Carlson, C. E., Devinsky, O., Doyle, W. K., Rubin, N., Heeger, D. J., & Hasson, U. (2012) · Neuron · doi:10.1016/j.neuron.2012.08.011 · Cited by 532
Areas of brain activation in males and females during viewing of erotic film excerpts
Karama, S., Lecours, A. R., Leroux, J., Bourgouin, P., Beaudoin, G., Joubert, S., & Beauregard, M. (2002) · Human Brain Mapping · doi:10.1002/hbm.10014 · Cited by 522
Language in context: Emergent features of word, sentence, and narrative comprehension
Xu, J., Kemeny, S., Park, G., Frattali, C., & Braun, A. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.12.013 · Cited by 502
Iacoboni, M., Lieberman, M. D., Knowlton, B. J., Molnar-Szakacs, I., Moritz, M., Throop, C. J., & Fiske, A. P. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2003.11.013 · Cited by 459
Kanske, P., Böckler, A., Trautwein, F.-M., & Singer, T. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.07.082 · Cited by 421
Coupled neural systems underlie the production and comprehension of naturalistic narrative speech
Silbert, L. J., Honey, C. J., Simony, E., Poeppel, D., & Hasson, U. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1323812111 · Cited by 352
Attention during natural vision warps semantic representation across the human brain
Çukur, T., Nishimoto, S., Huth, A. G., & Gallant, J. L. (2013) · Nature Neuroscience · doi:10.1038/nn.3381 · Cited by 351
van Marle, H. J. F., Hermans, E. J., Qin, S., & Fernández, G. (2009) · Biological Psychiatry · doi:10.1016/j.biopsych.2009.05.014 · Cited by 317
Converging language streams in the human temporal lobe
Spitsyna, G., Warren, J. E., Scott, S. K., Turkheimer, F. E., & Wise, R. J. S. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.0559-06.2006 · Cited by 312
Shared understanding of narratives is correlated with shared neural responses
Nguyen, M., Vanderwal, T., & Hasson, U. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2018.09.010 · Cited by 304 · cited 1×
“…that accumulates on larger timescales depends on DMN structures ( AbdulSabur et al., 2014 ; Babajani-Feremi, 2017 ; Gordon et al., 2017 ; Nguyen et al., 2019 ; Regev et al., 2013 ; Simony et al., 2016 ; Tikka et al., 2018 ; Wilson et al., 2008 ; Xu et al., 2005 ; Yarkoni et al., 2008b ;…”
Same Story, Different Story: The Neural Representation of Interpretive Frameworks
Yeshurun, Y., Swanson, S., Simony, E., Chen, J., Lazaridi, C., Honey, C. J., & Hasson, U. (2017) · Psychological Science · doi:10.1177/0956797616682029 · Cited by 300 · cited 1×
“Also, prenatal ocaine exposure has been observed to modulate responses to food cues n the ventral striatum, ACC, and MPFC, with this effect correlated with elf-reported food cravings ( Yip et al., 2014 ).”
BackgroundWhat constitutes an episode in episodic memory?
Ezzyat, Y., & Davachi, L. (2011) · Psychological Science · doi:10.1177/0956797610393742 · Cited by 298
Human brain activity time-locked to narrative event boundaries
Speer, N. K., Zacks, J. M., & Reynolds, J. R. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01920.x · Cited by 296
Zadbood, A., Chen, J., Leong, Y. C., Norman, K. A., & Hasson, U. (2017) · Cerebral Cortex · doi:10.1093/cercor/bhx202 · Cited by 281
Zadbood, A., Chen, J., Leong, Y. C., Norman, K. A., & Hasson, U. (2016) · doi:10.1101/081208 · Cited by 281
Beyond superior temporal cortex: Intersubject correlations in narrative speech comprehension
Wilson, S. M., Molnar-Szakacs, I., & Iacoboni, M. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhm049 · Cited by 281
van Marle, H. J. F., Hermans, E. J., Qin, S., & Fernández, G. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.05.070 · Cited by 251
Ben-Yakov, A., & Dudai, Y. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.0702-11.2011 · Cited by 238 · cited 15×
“In another study, reduced functional connectivity patterns were identified between the caudate nucleus, TPJ, DLPFC, and cingulate cortex after certain types of emotional attachment narrative ( Krause et al., 2018 ), and EEG findings suggest that, while subjects remain aroused after hearing attachment narratives, they aff…”
BackgroundInfluentialAttention Strongly Modulates Reliability of Neural Responses to Naturalistic Narrative Stimuli
Ki, J. J., Kelly, S. P., & Parra, L. C. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.2942-15.2016 · Cited by 221 · cited 1×
“In one such study, enhanced ISC of EEG stimulus-evoked activity during meaningful auditory and audio-visual narratives was noted to covary with subjects’ attentional state ( Ki et al., 2016 ).”
Multivariate neural biomarkers of emotional states are categorically distinct
Kragel, P. A., & LaBar, K. S. (2015) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsv032 · Cited by 218
Selective and invariant neural responses to spoken and written narratives
Regev, M., Honey, C. J., Simony, E., & Hasson, U. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.1580-13.2013 · Cited by 215
Prediction error associated with the perceptual segmentation of naturalistic events
Zacks, J. M., Kurby, C. A., Eisenberg, M. L., & Haroutunian, N. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00078 · Cited by 202
Home reading environment and brain activation in preschool children listening to stories
Hutton, J. S., Horowitz-Kraus, T., Mendelsohn, A. L., DeWitt, T., Holland, S. K., & the C-MIND Authorship Consortium (2015) · Pediatrics · doi:10.1542/peds.2015-0359 · Cited by 189
Synchronous brain activity across individuals underlies shared psychological perspectives
Lahnakoski, J. M., Glerean, E., Jääskeläinen, I. P., Hyönä, J., Hari, R., Sams, M., & Nummenmaa, L. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.06.022 · Cited by 180
Self-citationInter-subject synchronization of prefrontal cortex hemodynamic activity during natural viewing
Jääskeläinen, I. P., Koskentalo, K., Balk, M. H., Autti, T., Kauramäki, J., Pomren, C., & Sams, M. (2008) · The Open Neuroimaging Journal · doi:10.2174/1874440000802010014 · Cited by 166 · cited 1×
“In this reearch area, methodology advances have been vital ( Dayan et al., 2018 ; az et al., 2012 ; Jaaskelainen et al., 2008 , 2016b ; Nummenmaa et al., 012 , 2014a ; Viinikainen et al., 2012 ; Wallentin et al., 2011 ).”
MethodSelf-citationNeural correlates of narrative shifts during auditory story comprehension
Whitney, C., Huber, W., Klann, J., Weis, S., Krach, S., & Kircher, T. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.04.037 · Cited by 156
Hippocampal immediate poststimulus activity in the encoding of consecutive naturalistic episodes
Ben-Yakov, A., Eshel, N., & Dudai, Y. (2013) · Journal of Experimental Psychology: General · doi:10.1037/a0033558 · Cited by 153 · cited 1×
“This hippocampal activity predicts recall accuracy (Ben-Yakov et al., 2013), with hippocampal activity being lower during presentation of familiar event sequences (Ben-Yakov et al.”
BackgroundDistributed affective space represents multiple emotion categories across the human brain
Saarimäki, H., Ejtehadian, L. F., Glerean, E., Jääskeläinen, I. P., Vuilleumier, P., Sams, M., & Nummenmaa, L. (2018) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsy018 · Cited by 146 · cited 2×
“Recent evidence indicates that it is possible to classify more specific emotions also, such as love and pride within the “happiness” basic-emotion category (Saarimäki et al., 2018).”
BackgroundSelf-citationFact vs fiction—how paratextual information shapes our reading processes
Altmann, U., Bohrn, I. C., Lubrich, O., Menninghaus, W., & Jacobs, A. M. (2014) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nss098 · Cited by 146 · cited 1×
“Interestingly, reading a passage labeled as fiction activated the precuneus more strongly than did reading the same passage as factual (Altmann et al., 2012).”
BackgroundLocalizing Pain Matrix and Theory of Mind networks with both verbal and non-verbal stimuli
Jacoby, N., Bruneau, E., Koster-Hale, J., & Saxe, R. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2015.11.025 · Cited by 141
Memorable Audiovisual Narratives Synchronize Sensory and Supramodal Neural Responses
Cohen, S. S., & Parra, L. C. (2016) · eneuro · doi:10.1523/eneuro.0203-16.2016 · Cited by 138 · cited 2×
“In another study, ISC of electroenephalographic (EEG) activity, estimated to originate in sensory and xtra-sensory areas, predicted subsequent memorization of audiovisual arratives ( Cohen and Parra, 2016 ).”
BackgroundNeural correlates of social cognition in naturalistic settings: a model-free analysis approach
Wolf, I., Dziobek, I., & Heekeren, H. R. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.08.060 · Cited by 137
Chen, J., Honey, C. J., Simony, E., Arcaro, M. J., Norman, K. A., & Hasson, U. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhv155 · Cited by 135
Changes in events alter how people remember recent information
Swallow, K. M., Barch, D. M., Head, D., Maley, C. J., Holder, D., & Zacks, J. M. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2010.21524 · Cited by 103
Multivoxel pattern analysis does not provide evidence to support the existence of basic emotions
Clark-Polner, E., Johnson, T. D., & Barrett, L. F. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhw028 · Cited by 97 · cited 1×
“, sadness always involving exactly the same replicable activation pattern (Clark-Polner et al., 2017).”
BackgroundAmplification of local changes along the timescale processing hierarchy
Yeshurun, Y., Nguyen, M., & Hasson, U. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1701652114 · Cited by 94 · cited 2×
“In other studies, individuals primed to hear a story from either of two distinct social perspectives (a protagonist with unfounded vs. justified jealousy) showed ISC differences in their DAN, hippocampus, DMN, mirror neuron system, and language-processing networks ( Yeshurun et al., 2017b ).”
Lehne, M., Engel, P., Rohrmeier, M., Menninghaus, W., Jacobs, A. M., & Koelsch, S. (2015) · PLOS ONE · doi:10.1371/journal.pone.0124550 · Cited by 84
Temporal scaling of neural responses to compressed and dilated natural speech
Lerner, Y., Honey, C. J., Katkov, M., & Hasson, U. (2014) · Journal of Neurophysiology · doi:10.1152/jn.00497.2013 · Cited by 75
Jääskeläinen, I. P., Pajula, J., Tohka, J., Lee, H.-J., Kuo, W.-J., & Lin, F.-H. (2016) · Scientific Reports · doi:10.1038/srep27741 · Cited by 52 · cited 4×
“In this reearch area, methodology advances have been vital ( Dayan et al., 2018 ; az et al., 2012 ; Jaaskelainen et al., 2008 , 2016b ; Nummenmaa et al., 012 , 2014a ; Viinikainen et al., 2012 ; Wallentin et al., 2011 ).”
BackgroundMethodInfluentialSelf-citationSaalasti, S., Alho, J., Bar, M., Glerean, E., Honkela, T., Kauppila, M., Sams, M., & Jääskeläinen, I. P. (2019) · Brain and Behavior · doi:10.1002/brb3.1288 · Cited by 50
Self-citationInterpretation-mediated changes in neural activity during language comprehension
Cooper, E. A., Hasson, U., & Small, S. L. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.01.003 · Cited by 31
Neural correlates of experimental trauma memory retrieval
Gvozdanovic, G. A., Stämpfli, P., Seifritz, E., & Rasch, B. (2017) · Human Brain Mapping · doi:10.1002/hbm.23613 · Cited by 23 · cited 1×
“…the precuneus, lateral temporal areas, MPFC, and hippocampus ( Mendelsohn et al., 2010 ; Risius et al., 2013 ). etrieval of memories associated with emotionally traumatic film, in urn, resulted in activation of precuneous, retrosplenial cortex, and cinulate cortex ( Gvozdanovic et al., 2017 ).”
Background
Cited by
Yilmaz, M. B., Lotman, E., Karjus, A., & Tikka, P. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1160843 · Cited by 23
Metadata
- Authors
- Jääskeläinen, I. P., Sams, M., Glerean, E., & Ahveninen, J.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2021-01
- Volume
- 224
- Pages
- 117445
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W3092752701
- Cited by
- 183
- References
- 58
- FWCI
- 9.14
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.