Discovering Event Structure in Continuous Narrative Perception and Memory

In the paper

“The hippocampus is thought to bind cortical representations into a memory trace (McClelland, McNaughton, & O’Reilly, 1995; Moscovitch et al., 2005; Norman & O’Reilly, 2003)”

What the sources say

• ?Not checked — no full text of the sourcecites McClelland et al., 1995 source
• Supported

  The Abstract states that the hippocampus binds neurons in a neocortical and limbic network into a memory trace. The passage headed “Multiple trace theory: an alternative to the standard model” likewise says the hippocampal complex binds neurons representing an experience into a memory trace. (LLM, conf=0.95)

  cites Moscovitch et al., 2005 source
• ?Not checked — no full text of the sourcecites Norman & O’Reilly, 2003 source

In the paper

“Recent work has shown that hippocampal activity peaks at the offset of video clips (Ben-Yakov & Dudai, 2011; Ben-Yakov, Eshel, & Dudai, 2013)”

What the sources say

• Partly supported

  The Experiment 2 results (page not specified in our copy) conclude that delayed activity in the bilateral hippocampus is time-locked to clip offset. The paper describes the response as beginning around or shortly after the clip ends, however, not as peaking at that moment. (LLM, conf=0.86)

  cites Ben-Yakov & Dudai, 2011 sourceExperimental — accuracy not yet measured
• ?Not checked — no full text of the sourcecites Ben-Yakov et al., 2013 source

In the paper

“Prior work has shown that the end of a video clip is associated with increased hippocampal activity, and the magnitude of the activity predicts later memory (Ben-Yakov & Dudai, 2011; Ben-Yakov et al., 2013).”

What the sources say

• Supported

  The Results describing Figure 3 report hippocampal activity beginning around clip offset, with a higher-amplitude response after clips that were later remembered than after those forgotten. The Experiment 2 results further identify subsequent-memory-related activity in the bilateral hippocampus as time-locked to clip offset. Page numbers are not provided in our copy. (LLM, conf=0.91)

  cites Ben-Yakov & Dudai, 2011 source
• ?Not checked — no full text of the sourcecites Ben-Yakov et al., 2013 source

In the paper

“Processing timescales increase from tens of milliseconds in early sensory regions (e.g. for detecting phonemes in early auditory areas), to a few seconds in mid-level sensory areas (e.g. for integrating words into sentences), up to hundreds of seconds in regions including the temporoparietal junction, angular gyrus, and posterior and frontal medial cortex”

What the sources say

• Partly supported

  The Introduction (page unavailable) describes a hierarchy from sensory regions with processing timescales shorter than a second to higher-order regions influenced by information from minutes earlier. The Discussion specifies hundreds of milliseconds for low-level sensory areas, not the claimed tens of milliseconds. Thus the hierarchy and long-timescale end are supported, but the precise short-timescale value is not. (LLM, conf=0.85)

  cites J. Chen et al., 2016 sourceExperimental — accuracy not yet measured
• Supported

  The Process memory framework section describes temporal receptive windows increasing from tens to hundreds of milliseconds, through several seconds, to tens to hundreds of seconds, and says they increase hierarchically from early sensory to higher-order areas. The hierarchical topography section identifies the temporoparietal junction and angular gyrus as long-timescale regions; the Process Memory and the Medial Temporal Lobe section also identifies posterior and medial prefrontal regions. The copy does not provide page numbers for these passages. (LLM, conf=0.90)

  cites Hasson, Chen, & Honey, 2015 source

In the paper

“The prior study used a hand-labeled event correspondence between these two stimuli to show that activity patterns evoked by corresponding events in the movie and narration were correlated in a network of regions including angular gyrus, precuneus, retrosplenial cortex, posterior cingulate cortex and mPFC (Zadbood et al., 2016).”

What the sources say

• Supported

  The Pattern Similarity Searchlight section says an independent rater defined 22 movie scenes and the corresponding scenes were identified in the spoken narration. The Pattern Similarity Between Listening and Movie-Viewing section reports significant event-specific correlations and names angular gyrus, precuneus, retrosplenial cortex, PCC, and mPFC. (LLM, conf=0.95)

  cites Zadbood et al., 2016 source

In the paper

“Behavioral experiments and mathematical models have argued that long-term memory reflects event structure during encoding (Ezzyat & Davachi, 2011; Gershman, Radulescu, Norman, & Niv, 2014; Sargent et al., 2013; Zacks, Tversky, et al., 2001)”

What the sources say

• Supported

  The Abstract reports that long-term associative memory was weaker across event boundaries than within events, suggesting that segmentation during encoding was reflected in memory. The General Discussion likewise states that event boundaries influenced long-term memory organization. (LLM, conf=0.94)

  cites Ezzyat & Davachi, 2011 source
• Supported

  The Discussion says that memory structure reflects how experience is parsed as the environment changes, with distinct memories formed at abrupt discontinuities. The Introduction likewise proposes that inputs are grouped into stored traces according to the underlying structure of experience. (LLM, conf=0.90)

  cites Gershman et al., 2014 source
• Supported

  The Introduction reports that long-term associations are stronger within events than between them and concludes that how experience is segmented affects how it is remembered. The Discussion also reports that event-segmentation ability uniquely predicted memory for the experience. Page numbers are not provided in our copy. (LLM, conf=0.91)

  cites Sargent et al., 2013 source

In the paper

“we demonstrated that event-related representations generalize across modalities and between encoding and recall (thereby supporting hypotheses 3 and 5, J. Chen et al., 2017; Zadbood, Chen, Leong, Norman, & Hasson, 2016)”

What the sources say

• Supported

  The Results (‘Neural reinstatement within participants’; page not available) report that scene-specific patterns from movie viewing were reinstated during spoken recall. The Results (‘Pattern similarity between participants’) describe this as recall of an audio-visual narrative, supporting generalization across modalities as well as from encoding to recall. (LLM, conf=0.92)

  cites J. Chen et al., 2017 source
• Supported

  The Pattern Similarity Between Spoken-Recall and Movie-Viewing section reports that scene-specific patterns from movie encoding were reinstated during spoken recall. The Discussion reports shared event-specific patterns during audiovisual encoding, spoken recall, and listeners’ construction of the verbally described events, supporting generalization across those modalities and stages. The supplied passages do not specify page numbers. (LLM, conf=0.92)

  cites Zadbood et al., 2016 source

In the paper

“However, these methods do not explicitly model event boundaries.”

What the sources say

• ?Not checked — no full text of the sourcecites Kang & Wheatley, 2015 source
• Partly supported

  The Time-Warp Analysis section says the paper’s time-warping procedure aligns recordings using similarities between sound waveforms, rather than describing an explicit model of event boundaries. It does not address conditional random fields, so it cannot support that part of the claim. (LLM, conf=0.75)

  cites Silbert, Honey, Simony, Poeppel, & Hasson, 2014 sourceExperimental — accuracy not yet measured
• Partly supported

  Section 4.4 (p. 6) defines the paper’s CRF in terms of shot-to-sentence alignments and consistency between nearby alignments, with no explicit event-boundary variable. It describes dynamic time warping only as enforcing forward-in-time matching; that description does not establish whether the dynamic-time-warping method models event boundaries. (LLM, conf=0.76)

  cites Zhu et al., 2015 sourceExperimental — accuracy not yet measured

In the paper

“These areas are involved in high-level, multimodal scene processing tasks including memory and navigation (Baldassano, Esteva, Beck, & Fei-Fei, 2016; Kumar, Federmeier, Fei-Fei, & Beck, 2017), are part of the “general recollection network” with strong anatomical and functional connectivity to the hippocampus (Rugg & Vilberg, 2013), and are the core components of the posterior medial memory system”

What the sources say

• Partly supported

  Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  cites Baldassano et al., 2016 sourceExperimental — accuracy not yet measured
• ?Not checked — no full text of the sourcecites Kumar et al., 2017 source
• Supported

  Split into 2 atoms — 2 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  cites Rugg & Vilberg, 2013 sourceExperimental — accuracy not yet measured

In the paper

“Previous work has shown that hand-labeled event boundaries are associated with univariate activity increases in a network of regions overlapping our high-level areas (Ezzyat & Davachi, 2011; Speer, Zacks, & Reynolds, 2007; Swallow et al., 2011; Whitney et al., 2009; Zacks, Braver, et al., 2001; Zacks, Speer, Swallow, & Maley, 2010)”

What the sources say

• Partly supported

  Experiment 2 Results (page unavailable) report greater activity for boundary than control sentences in the right posterior cingulate/precuneus, plus sentence-type-dependent responses in other regions. But Experiment 1 Method (page unavailable) says the boundaries were cued by experimentally inserted “A while later” phrases and checked by an independent group, not hand-labeled event boundaries. The paper therefore supports a boundary-related activity finding, but not that methodological description. (LLM, conf=0.86)

  cites Ezzyat & Davachi, 2011 sourceExperimental — accuracy not yet measured
• ?Not checked — no full text of the sourcecites Speer, Zacks, & Reynolds, 2007 source
• Partly supported

  The Materials and Methods (Materials; page unavailable) describe event boundaries identified from observers’ button presses. The Results (Imaging Data; page unavailable) report greater activity in medial temporal and parietal regions when objects were retrieved across an event boundary, but this is a retrieval-related effect, not a demonstrated network-wide increase at the hand-labeled boundaries themselves. (LLM, conf=0.84)

  cites Swallow et al., 2011 sourceExperimental — accuracy not yet measured

In the paper

“Other analyses of these datasets also found reactivation during recall (J. Chen et al., 2017) and shared event structure across modalities (Zadbood et al., 2016)”

What the sources say

• Supported

  The citation to Chen et al. supports the reactivation-during-recall portion of the sentence; the cross-modal event-structure portion is attributed to Zadbood et al. In the Discussion, Chen et al. report that movie-viewing neural patterns were reactivated during free verbal recollection. (LLM, conf=0.94)

  cites J. Chen et al., 2017 source
• Supported

  The section “Shared Neural Response Across 3 Conditions (Triple Shared Pattern Analysis)” reports shared event-specific neural patterns during movie viewing, spoken recall, and listening. The Discussion also describes shared patterns between audiovisual perception and mental construction from a verbal description. These findings support the cited clause about shared event structure across modalities; the separate recall-reactivation clause cites Chen et al. (2017). (LLM, conf=0.90)

  cites Zadbood et al., 2016 source

In the paper

“The “process memory framework” argues that perceptual stimuli are integrated across longer and longer timescales along a hierarchy from early sensory regions to regions in the default mode network.”

What the sources say

• Supported

  The Process memory framework section says temporal receptive windows increase hierarchically from early sensory to higher-order areas as incoming information is integrated over progressively longer periods. The Process Memory and the Medial Temporal Lobe section adds that areas with the longest timescales overlap broadly with the default mode network. (LLM, conf=0.94)

  cites Hasson et al., 2015 source

In the paper

“this topography has previously been mapped either by temporally scrambling the stimulus at different timescales to see which regions’ responses are disrupted (Hasson, Yang, Vallines, Heeger, & Rubin, 2008; Honey et al., 2012; Lerner, Honey, Silbert, & Hasson, 2011) or by examining the power spectrum of intrinsic dynamics within each region (Honey et al., 2012; Murray et al., 2014; Stephens et al.,”

What the sources say

• Supported

  The Abstract describes mapping a cortical hierarchy of temporal receptive windows by comparing responses to films scrambled at different time scales. The Intersubject correlation analysis reports that posterior cortex responded reliably across scrambling scales, whereas several higher-level regions did so only at longer scales. This citation supports the stimulus-scrambling part of the sentence; the power-spectrum method is attributed to other citations. Page numbers were not provided in our copy. (LLM, conf=0.93)

  cites Hasson et al., 2008 source
• Supported

  The Results (‘Topography of TRWs’; page not provided) define a regional temporal-receptive-window index by comparing responses to intact and fine-scrambled movie clips. The Experimental Procedures (‘Computing LowFq’) describe analyzing power spectra, including during fixation, and the Results (‘Dynamic Timescales Measured During Fixation Predict TRWs’) link those dynamics to the regional index. (LLM, conf=0.92)

  cites Honey et al., 2012 source
• Supported

  The Introduction says the study scrambled a narrated story at word, sentence, and paragraph timescales to assess responses across brain areas. The Results and Figure 2 describe maps of response reliability for each scrambling condition. Lerner et al. is cited for the scrambling method, not the separate power-spectrum method. (LLM, conf=0.93)

  cites Lerner et al., 2011 source

In the paper

“Sensory regions like visual and auditory cortex showed faster transitions between stable activity patterns, while higher-level regions like the posterior medial cortex, angular gyrus, and intraparietal sulcus had activity patterns that often remained constant for over a minute before transitioning to a new stable pattern”

In the paper

“we found significant model fits in both the posterior cingulate (p=0.015) and the angular gyrus (p=0.002), but not in low-level auditory cortex (p=0.277) (Fig. 7b).”

In the paper

“The number of event boundaries identified by the observers varied between 36 and 64, but the boundaries had a significant amount of overlap, with an average pairwise Dice’s coefficient of 0.63 and 20 event boundaries that were labeled by all four raters.”

In the paper

“all regions showed an above-chance match to human annotations (early visual, p=0.0135; late visual, p=0.0065; angular gyrus, p=0.0011; posterior medial, p<0.001), but this match increased across the layers of the hierarchy and was largest in angular gyrus and posterior medial cortex”

In the paper

“We found that narrative stimuli evoke event-structured activity throughout the cortex, with pattern dynamics consisting of relatively stable periods punctuated by rapid event transitions.”

In the paper

“we found that encoding activity and event reactivation were positively correlated in both angular gyrus (r=0.362, p=0.002) and the posterior cingulate (r=0.312, p=0.042), but not early auditory cortex (r=0.080, p=0.333).”

In the paper

“In a network of high-level association regions, we found that these events were related to subjective event annotations by human observers”

In the paper

“there was no relationship between the hippocampal activity at the starting boundary of an event and that event’s later reinstatement in the angular gyrus (r=−0.119, p=0.867; difference from ending boundary correlation p=0.004) and only a weak, nonsignificant relationship in posterior cingulate (r=0.189, p=0.113; difference from ending boundary correlation p=0.274).”

In the paper

“Note that both the event boundaries and the hippocampal response are hemodynamic signals, so there is no hemodynamic offset between these two measures.”

In the paper

“flexibly adjust their units of segmentation depending on their uncertainty about ongoing events (Newtson, 1973)”

What the sources say

• ?Not checked — no full text of the sourcecites Newtson, 1973 source

In the paper

“presumably because the low level auditory features of the two stimuli were markedly different.”

In the paper

“The hippocampal response does start slightly before the event boundary, which could be due to uncertainty in the model estimation of the exact boundary timepoint and/or anticipation that the event is about to end.”

In the paper

“As shown in Fig. S2, our method was highly effective at identifying events, with the majority of the identified boundaries falling close to a story switch.”

In the paper

“We also conducted an alternate version of the segmentation analysis that does not rely on precise MNI alignment (P.-H. (Cameron) Chen et al., 2015) and obtained similar results (see Fig. S4c).”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites P.-H. (Cameron) Chen et al., 2015

In the paper

“We restricted our searchlight analyses to voxels that were reliably driven by the stimuli, measured using intersubject correlation (Hasson, Nir, Levy, Fuhrmann, & Malach, 2004).”

What the sources say

• ?Not checked — no full text of the sourcecites Hasson, Nir, Levy, Fuhrmann, & Malach, 2004 source

In the paper

“Permutation or resampling analyses were used to statistically evaluate all of the results.”

In the paper

“The power spectrum is therefore an incomplete description of voxel dynamics, since the correlation timescale changes dynamically, with faster changes at event boundaries and slower changes within boundaries (see also Fig. S4b).”

In the paper

“The model also is not fMRI-specific”

In the paper

“Furthermore, the angular gyrus and posterior medial cortex exhibit a set of overlapping properties associated with high-level situation model representations: long event timescales, event boundaries closely related to human annotations, generalization across modalities, hippocampal response at event boundaries, reactivation during free recall, and anticipatory coding for familiar narratives.”

In the paper

“We measured the robustness of the identified boundaries by testing whether these boundaries explained the data in the held-out subject.”

In the paper

“these events are integrated along the processing hierarchy into meaningful, temporally extended, episodic structures, to be later encoded into memory via interaction with the hippocampus.”

In the paper

“A similar result can be obtained by directly aligning the two listening conditions without reference to the movie-watching condition (see Fig. S7).”

In the paper

“suggesting that all processing regions can exhibit rapid activity shifts, but that these fast changes are much less frequent in long timescale regions.”

In the paper

“Data from 3 subjects were discarded due to falling asleep during the scan, and 1 due to problems with audio delivery.”