Constructing Experience: Event Models from Perception to Action
Open accessThe DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Mental representations of everyday experience are rich, structured, and multimodal. In this article, we consider the adaptive pressures that led to humans’ construction of such representations, arguing that structured event representations enable cognitive systems to more effectively predict the trajectory of naturalistic everyday activity. We propose an account of how cortical systems and the hippocampus interact to construct, maintain, and update event representations. This analysis throws light on recent research on story comprehension, event segmentation, episodic memory, and action planning. It also suggests how the growing science base can be deployed to diagnose impairments in event perception and memory, and to improve memory for everyday events.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The episodic buffer: a new component of working memory?
Baddeley, A. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01538-2 · Cited by 7,359 · cited 7×
“Studies of complex task performance in monkeys and humans suggest that the PFC represents the structure of one’s current task simultaneously on multiple timescales, possibly with a temporal gradient such that shorter timescales are represented more posteriorly and longer timescales are represented more anteriorly [93–…”
BackgroundResultInfluentialWhatever next? Predictive brains, situated agents, and the future of cognitive science
Clark, A. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12000477 · Cited by 5,924 · cited 2×
“The magnitude of prediction error, then, is given by the sum of squared error between pt and st+1 (in reinforcement learning models, this quantity is referred to as ‘surprisal’ [6]).”
BackgroundMethodMemory and the hippocampus: A synthesis from findings with rats, monkeys, and humans.
Squire, L. R. (1992) · Psychological Review · doi:10.1037/0033-295x.99.2.195 · Cited by 5,376
Plans and the structure of behavior.
Miller, G. A., Galanter, E., & Pribram, K. H. (1960) · doi:10.1037/10039-000 · Cited by 4,630
Harnad, S. (1990) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(90)90087-6 · Cited by 4,235 · cited 1×
“– to be useful, it needs to be closely coupled to the evolving state of affairs in the real world [16].”
BackgroundThe processes of causal attribution.
Kelley, H. H. (1973) · American Psychologist · doi:10.1037/h0034225 · Cited by 4,025 · cited 1×
“There is good reason to think that actors’ and observers’ representations of an activity often differ substantially because actors and observers have access to different features of the activity [51,52].”
BackgroundEricsson, K. A., & Kintsch, W. (1995) · Psychological Review · doi:10.1037/0033-295x.102.2.211 · Cited by 3,498 · cited 1×
“As Baddeley noted, such a representation probably leverages long-term memory to boost its capacity and durability (cf [70]).”
BackgroundSituation models in language comprehension and memory
Zwaan, R. A., & Radvansky, G. A. (1998) · Psychological Bulletin · doi:10.1037/0033-2909.123.2.162 · Cited by 2,877
Buckner, R. L., & Carroll, D. C. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.11.004 · Cited by 2,691
Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex
Funahashi, S., Bruce, C. J., & Goldman-Rakic, P. S. (1989) · Journal of Neurophysiology · doi:10.1152/jn.1989.61.2.331 · Cited by 2,685
The Medial Temporal Lobe and Recognition Memory
Eichenbaum, H., Yonelinas, A. P., & Ranganath, C. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.30.051606.094328 · Cited by 2,580
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Neuron Activity Related to Short-Term Memory
Fuster, J. M., & Alexander, G. E. (1971) · Science · doi:10.1126/science.173.3997.652 · Cited by 2,085
DEFICITS IN STRATEGY APPLICATION FOLLOWING FRONTAL LOBE DAMAGE IN MAN
SHALLICE, T., & BURGESS, P. W. (1991) · Brain · doi:10.1093/brain/114.2.727 · Cited by 2,017 · cited 6×
“Studies of complex task performance in monkeys and humans suggest that the PFC represents the structure of one’s current task simultaneously on multiple timescales, possibly with a temporal gradient such that shorter timescales are represented more posteriorly and longer timescales are represented more anteriorly [93–…”
BackgroundInfluentialError Correction, Sensory Prediction, and Adaptation in Motor Control
Shadmehr, R., Smith, M. A., & Krakauer, J. W. (2010) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-060909-153135 · Cited by 1,869 · cited 1×
“One thing that is striking about these literatures is how powerful simple learning mechanisms can be – prediction has been observed in diverse domains including classical conditioning [4], visual feature detection [7], motion perception [8], motor control [9], scene perception [10], and language comprehension [11].”
BackgroundPlace units in the hippocampus of the freely moving rat
O'Keefe, J. (1976) · Experimental Neurology · doi:10.1016/0014-4886(76)90055-8 · Cited by 1,836
Brain mechanisms linking language and action
Pulvermüller, F. (2005) · Nature Reviews Neuroscience · doi:10.1038/nrn1706 · Cited by 1,828
Neuronal Coding of Prediction Errors
Schultz, W., & Dickinson, A. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.473 · Cited by 1,427 · cited 1×
“fMRI indicated that these conditions were associated with greater activity in the midbrain dopamine system which is involved in signaling prediction errors [59].”
BackgroundThe theory of constructed emotion: an active inference account of interoception and categorization
Barrett, L. F. (2016) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsw154 · Cited by 1,426
The future of memory: remembering, imagining, and the brain
Schacter, D. L., Addis, D. R., Hassabis, D., Martin, V. C., Spreng, R. N., & Szpunar, K. K. (2012) · Neuron · doi:10.1016/j.neuron.2012.11.001 · Cited by 1,398
Two cortical systems for memory-guided behaviour
Ranganath, C., & Ritchey, M. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3338 · Cited by 1,377 · cited 2×
“At an event boundary, the HPC may perform a ‘now print’ [85–87] or ‘sharpening’ function [82] that stabilizes the bound pattern in a way that preserves it for subsequent recall, before abandoning the current binding to make room for binding a new working model (Figure 2).”
BackgroundDeconstructing episodic memory with construction
Hassabis, D., & Maguire, E. A. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.05.001 · Cited by 1,229 · cited 1×
“Recalling something from the past or imagining something in the future appears to trigger a common scene construction mechanism, resulting in a structured mental representation complete with spatiotemporal context and vivid semantic and sensory details [26,27].”
BackgroundHippocampal ‘time cells’ bridge the gap in memory for discontiguous events
MacDonald, C. J., Lepage, K. Q., Eden, U. T., & Eichenbaum, H. (2011) · Neuron · doi:10.1016/j.neuron.2011.07.012 · Cited by 1,193 · cited 1×
“Building on work on place cells in the HPC [45], and on work on HPC representations of temporal context in human verbal memory [46,47], Eichenbaum and colleagues used electrophysiology in rodents to characterize time cells which respond to specific timepoints within a temporal sequence [46].”
BackgroundEvent perception: a mind-brain perspective
Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S., & Reynolds, J. R. (2007) · Psychological Bulletin · doi:10.1037/0033-2909.133.2.273 · Cited by 1,148
Self-citationThe proactive brain: using analogies and associations to generate predictions
Bar, M. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.05.005 · Cited by 1,144
Raichle, M. E. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.01.008 · Cited by 1,111 · cited 2×
“Moreover, the brain appears to spend a substantial amount of its energy budget on predictive processing that puts it in a position to respond to predicted environmental features [3].”
BackgroundThe theory of constructed emotion: an active inference account of interoception and categorization
Barrett, L. F. (2017) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsx060 · Cited by 1,009 · cited 1×
“Barrett and colleagues have proposed a theory of emotional experience that sees affect as a component of a predictive internal model, and this theory has much in common with working models as described here [54,55].”
ResultCognitive control, hierarchy, and the rostro–caudal organization of the frontal lobes
Badre, D. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.004 · Cited by 980
Discovering Event Structure in Continuous Narrative Perception and Memory
Baldassano, C., Chen, J., Zadbood, A., Pillow, J. W., Hasson, U., & Norman, K. A. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.041 · Cited by 952
Sentence memory: A constructive versus interpretive approach
Bransford, J. D., Barclay, J. R., & Franks, J. J. (1972) · Cognitive Psychology · doi:10.1016/0010-0285(72)90003-5 · Cited by 908 · cited 1×
“Situation models help the reader to make predictions about relations between elements in a text [17].”
BackgroundMultiple routes to memory: distinct medial temporal lobe processes build item and source memories
Davachi, L., Mitchell, J. P., & Wagner, A. D. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0337195100 · Cited by 899
A hierarchy of temporal receptive windows in human cortex
Hasson, U., Yang, E., Vallines, I., Heeger, D. J., & Rubin, N. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.5487-07.2008 · Cited by 874
Segmentation in the perception and memory of events
Kurby, C. A., & Zacks, J. M. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.11.004 · Cited by 776
Self-citationPerceiving, remembering, and communicating structure in events.
Zacks, J. M., Tversky, B., & Iyer, G. (2001) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.130.1.29 · Cited by 640 · cited 2×
“This subjective experience corresponds with phasic activity throughout the cortex [32], and can be easily and reproducibly captured by asking people to mark off the boundaries as they occur [33,34].”
BackgroundSelf-citationPerceptions of perceptual symbols
Barsalou, L. W. (1999) · Behavioral and Brain Sciences · doi:10.1017/s0140525x99532147 · Cited by 620
Neural representations of events arise from temporal community structure
Schapiro, A. C., Rogers, T. T., Cordova, N. I., Turk-Browne, N. B., & Botvinick, M. M. (2013) · Nature Neuroscience · doi:10.1038/nn.3331 · Cited by 556
Event boundaries in perception affect memory encoding and updating
Swallow, K. M., Zacks, J. M., & Abrams, R. A. (2009) · Journal of Experimental Psychology: General · doi:10.1037/a0015631 · Cited by 382 · cited 1×
“The first studies to investigate event-based memory updating in dynamic audiovisual events used clips from commercial cinema, which were pre-tested to identify points that were widely judged to be event boundaries [64,65].”
BackgroundSelf-citationThe dependence of span and delayed-response performance on prefrontal cortex.
D’Esposito, M., & Postle, B. R. (1999) · Neuropsychologia · doi:10.1016/s0028-3932(99)00021-4 · Cited by 350
What constitutes an episode in episodic memory?
Ezzyat, Y., & Davachi, L. (2011) · Psychological Science · doi:10.1177/0956797610393742 · Cited by 298
Human brain activity time-locked to narrative event boundaries
Speer, N. K., Zacks, J. M., & Reynolds, J. R. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01920.x · Cited by 296 · cited 1×
“Studies of the phasic response of the brain at event boundaries have provided evidence that some resource-intensive activity takes place at the points where readers and viewers identify event boundaries [32,41,42].”
BackgroundSelf-citationBen-Yakov, A., & Dudai, Y. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.0702-11.2011 · Cited by 238
Temporal binding within and across events
DuBrow, S., & Davachi, L. (2016) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2016.07.011 · Cited by 226
Event segmentation ability uniquely predicts event memory
Sargent, J. Q., Zacks, J. M., Hambrick, D. Z., Zacks, R. T., Kurby, C. A., Bailey, H. R., Eisenberg, M. L., & Beck, T. M. (2013) · Cognition · doi:10.1016/j.cognition.2013.07.002 · Cited by 223
Self-citationPrediction error associated with the perceptual segmentation of naturalistic events
Zacks, J. M., Kurby, C. A., Eisenberg, M. L., & Haroutunian, N. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00078 · Cited by 202
Self-citationHippocampal immediate poststimulus activity in the encoding of consecutive naturalistic episodes
Ben-Yakov, A., Eshel, N., & Dudai, Y. (2013) · Journal of Experimental Psychology: General · doi:10.1037/a0033558 · Cited by 153
Contextual Processing in Episodic Future Thought
Szpunar, K. K., Chan, J. C. K., & McDermott, K. B. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn191 · Cited by 132 · cited 1×
“Recalling something from the past or imagining something in the future appears to trigger a common scene construction mechanism, resulting in a structured mental representation complete with spatiotemporal context and vivid semantic and sensory details [26,27].”
BackgroundMüller, N. G., Machado, L., & Knight, R. T. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/08989290260138582 · Cited by 130
Changes in events alter how people remember recent information
Swallow, K. M., Barch, D. M., Head, D., Maley, C. J., Holder, D., & Zacks, J. M. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2010.21524 · Cited by 103 · cited 2×
“The first studies to investigate event-based memory updating in dynamic audiovisual events used clips from commercial cinema, which were pre-tested to identify points that were widely judged to be event boundaries [64,65].”
BackgroundSelf-citation
Cited by
Inductive biases for deep learning of higher-level cognition
Goyal, A., & Bengio, Y. (2022) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2021.0068 · Cited by 300
Structured Event Memory: A neuro-symbolic model of event cognition.
Franklin, N. T., Norman, K. A., Ranganath, C., Zacks, J. M., & Gershman, S. J. (2020) · Psychological Review · doi:10.1037/rev0000177 · Cited by 177
Self-citationHasson, U., Egidi, G., Marelli, M., & Willems, R. M. (2018) · Cognition · doi:10.1016/j.cognition.2018.06.018 · Cited by 157
Neural Mechanisms of Episodic Retrieval Support Divergent Creative Thinking
Madore, K. P., Thakral, P. P., Beaty, R. E., Addis, D. R., & Schacter, D. L. (2019) · Cerebral Cortex · doi:10.1093/cercor/bhx312 · Cited by 137
Hippocampal neurons represent events as transferable units of experience
Sun, C., Yang, W., Martin, J., & Tonegawa, S. (2020) · Nature Neuroscience · doi:10.1038/s41593-020-0614-x · Cited by 135
Metadata
- Authors
- Richmond, L. L., & Zacks, J. M.
- Venue
- Trends in Cognitive Sciences
- Publisher
- Elsevier BV
- Published
- 2017-12
- Volume
- 21
- Issue
- 12
- Pages
- 962-980
- ISSN
- 1364-6613 (ISSN-L)
- OpenAlex
- W2750911160
- Cited by
- 188
- References
- 48
- FWCI
- 2.56
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.