Hippocampal replay of extended experience

“Place cells in the hippocampal formation fire selectively when an animal moves through particular locations (‘‘place fields’’) in the environment (O’Keefe and Dostrovsky, 1971; Wilson and McNaughton, 1993).”

“…replay detection in the decoded spatial domain affords advantages over methods that examine firing order across individual units, such as pair-wise correlation (Wilson and McNaughton, 1994) and spike-sequence detection (Diba and Buzsáki, 2007; Foster and Wilson, 2006; Lee and Wilson, 2004).”

“This interpretation of the functional role of awake replay is also consistent with work suggesting a high degree of overlap in the cognitive processes supporting both episodic recall and the evaluation of future events in humans (Buckner and Carroll, 2007; Schacter et al., 2007).”

“Place cells in the hippocampal formation fire selectively when an animal moves through particular locations (‘‘place fields’’) in the environment (O’Keefe and Dostrovsky, 1971; Wilson and McNaughton, 1993).”

“For example, the reverse replay event in Figure 4D proceeds in the B/A direction (from 7.5 to 2.5 m on the track) but uses the hippocampal ensemble code associated with running in the opposite A/B direction, as indicated by the blue color.”

“Previous studies have taken advantage of the joint tuning of many CA1 cells to running direction and location on the linear track (McNaughton et al., 1983; Muller et al., 1994) to demonstrate that spiking sequences can be replayed in either the forward or reverse temporal order (Diba and Buzsáki, 2007; Foster and Wil…”

“Hippocampal replay has been proposed to contribute to memory consolidation during sleep (Buzsáki, 1989; Marshall and Born, 2007; Stickgold, 2005).”

“To determine if significant replay events represented forward or reverse replay, we computed a ‘‘replay order’’ score as the difference between the mean likelihoods that the estimated running direction on the trajectory was in the same or opposite direction as the actual replayed trajectory (see Supplemental Experimen…”

“Such trains have been noted since the first reports of ripples (Buzsáki et al., 1983; O’Keefe and Nadel, 1978; Suzuki and Smith, 1987), but no function has previously been ascribed to this phenomenon.”

“…occur during periodic increases in neocortical population activity (‘‘up states’’) (Battaglia et al., 2004; Clemens et al., 2007; Mölle et al., 2006; Sirota et al., 2003) associated with slow oscillations in the cortical EEG (Isomura et al., 2006; Steriade, 2006; Wolansky et al., 2006).”

“In this respect, awake replay is similar to sequence reactivation during slow-wave sleep (Ji and Wilson, 2007; Lee and Wilson, 2002).”

“During pauses in locomotion and during slow-wave sleep, many place cells are recruited in intermittent population bursts, which are accompanied by ripples in the hippocampal local field potential (Buzsáki et al., 1992; Chrobak and Buzsáki, 1996).”

“There are significantly more remote replay events than the number of false positives expected to be generated by our replay detection procedure (52 of 1656 events; p < 10 11 under a binomial distribution, using false-positive rate of 1%).”

“These effects each lasted 50 ms, which is comparable to the duration of single sharp-wave ripple complexes as described previously (Ylinen et al., 1995).”

“Previous studies have taken advantage of the joint tuning of many CA1 cells to running direction and location on the linear track (McNaughton et al., 1983; Muller et al., 1994) to demonstrate that spiking sequences can be replayed in either the forward or reverse temporal order (Diba and Buzsáki,…”

“…sleep, cortical inputs may bias or otherwise influence memory reactivation (Ji and Wilson, 2007), given the complex bidirectional interactions between the hippocampus and neocortex (Isomura et al., 2006; Mölle et al., 2006; Siapas and Wilson, 1998; Sirota et al., 2003; Wolansky et al., 2006).”

“…with weak place-related firing (peak rate in tuning curve <3 Hz), leaving 47, 34, 23, 32, and 21 spatially tuned single units in rats 1–5 (cluster quality measures: L ratio 0.12 ± 0.17; isolation distance 17 ± 9; calculated using peak amplitude on each channel [Schmitzer-Torbert et al., 2005]).”

“We reasoned that during reactivation of previous experience the position estimate would deviate systematically from the actual position (Johnson and Redish, 2007).”

“Replay events have consistently been found to co-occur with ripple oscillations in the hippocampal local field potential (Diba and Buzsáki, 2007; Foster and Wilson, 2006; Ji and Wilson, 2007; Lee and Wilson, 2002; Nádasdy et al., 1999).”

“…sleep, cortical inputs may bias or otherwise influence memory reactivation (Ji and Wilson, 2007), given the complex bidirectional interactions between the hippocampus and neocortex (Isomura et al., 2006; Mölle et al., 2006; Siapas and Wilson, 1998; Sirota et al., 2003; Wolansky et al., 2006).”

“We also observed many significant replay trajectories that began at remote locations (Figure 3F; see examples in Figures 3B, 4C, and 4D).”

“…replayed spatial memory sequences, we employed a probabilistic neural decoding strategy to estimate the animal’s position on the track from the ensemble of spike trains (Brown et al., 1998; Wilson and Neuron 63, 497–507, August 27, 2009 ª2009 Elsevier Inc. 497 McNaughton, 1993; Zhang et al., 1998).”

“Hippocampal replay has been proposed to contribute to memory consolidation during sleep (Buzsáki, 1989; Marshall and Born, 2007; Stickgold, 2005).”

“There are significantly more remote replay events than the number of false positives expected to be generated by our replay detection procedure (52 of 1656 events; p < 10 11 under a binomial distribution, using false-positive rate of 1%).”

“During pauses in locomotion and during slow-wave sleep, many place cells are recruited in intermittent population bursts, which are accompanied by ripples in the hippocampal local field potential (Buzsáki et al., 1992; Chrobak and Buzsáki, 1996).”

“…replayed spatial memory sequences, we employed a probabilistic neural decoding strategy to estimate the animal’s position on the track from the ensemble of spike trains (Brown et al., 1998; Wilson and Neuron 63, 497–507, August 27, 2009 ª2009 Elsevier Inc. 497 McNaughton, 1993; Zhang et al., 1998).”

“One possible mechanism for the generation of trains of subsequences is provided by the re-entrant loops in the hippocampal-entorhinal circuitry (Canto et al., 2008; Kloosterman et al., 2004).”

“Interestingly, groups of ripples are also present during slowwave sleep, where they predominantly occur during periodic increases in neocortical population activity (‘‘up states’’) (Battaglia et al., 2004; Clemens et al., 2007; Mölle et al., 2006; Sirota et al., 2003) associated with slow oscillations in the cortical…”

“…sleep, cortical inputs may bias or otherwise influence memory reactivation (Ji and Wilson, 2007), given the complex bidirectional interactions between the hippocampus and neocortex (Isomura et al., 2006; Mölle et al., 2006; Siapas and Wilson, 1998; Sirota et al., 2003; Wolansky et al., 2006).”

“…studies have taken advantage of the joint tuning of many CA1 cells to running direction and location on the linear track (McNaughton et al., 1983; Muller et al., 1994) to demonstrate that spiking sequences can be replayed in either the forward or reverse temporal order (Diba and Buzsáki, 2007;…”

“…are also present during slowwave sleep, where they predominantly occur during periodic increases in neocortical population activity (‘‘up states’’) (Battaglia et al., 2004; Clemens et al., 2007; Mölle et al., 2006; Sirota et al., 2003) associated with slow oscillations in the cortical EEG…”

“…sleep, cortical inputs may bias or otherwise influence memory reactivation (Ji and Wilson, 2007), given the complex bidirectional interactions between the hippocampus and neocortex (Isomura et al., 2006; Mölle et al., 2006; Siapas and Wilson, 1998; Sirota et al., 2003; Wolansky et al., 2006).”

“…structure of replayed trajectories in a statistically rigorous manner and makes optimal use of the spatial information contained in hippocampal spikes, including the activity of cells with irregularly shaped place fields (such as those with multiple firing fields [Fenton et al., 2008]; Figure S8).”

“This last possibility, while speculative, is lent some support by the recent finding of specific reactivation of hippocampal neurons during free recall in humans (GelbardSagiv et al., 2008) and by specific activation of the hippocampus during sequence recall tasks in humans (Lehn et al., 2009).”

“There are significantly more remote replay events than the number of false positives expected to be generated by our replay detection procedure (52 of 1656 events; p < 10 11 under a binomial distribution, using false-positive rate of 1%).”

“We used a variation of Skaggs’ (Skaggs et al., 2007) ripple-detection procedure, which allows for the detection of closely spaced ripples.”

“Such trains have been noted since the first reports of ripples (Buzsáki et al., 1983; O’Keefe and Nadel, 1978; Suzuki and Smith, 1987), but no function has previously been ascribed to this phenomenon.”

“One possible mechanism for the generation of trains of subsequences is provided by the re-entrant loops in the hippocampal-entorhinal circuitry (Canto et al., 2008; Kloosterman et al., 2004).”

“Previous studies have taken advantage of the joint tuning of many CA1 cells to running direction and location on the linear track (McNaughton et al., 1983; Muller et al., 1994) to demonstrate that spiking sequences can be replayed in either the forward or reverse temporal order (Diba and Buzsáki, 2007; Foster and Wil…”

“…replay detection in the decoded spatial domain affords advantages over methods that examine firing order across individual units, such as pair-wise correlation (Wilson and McNaughton, 1994) and spike-sequence detection (Diba and Buzsáki, 2007; Foster and Wilson, 2006; Lee and Wilson, 2004).”

“In larger experimental environments, both forward and reverse replay can occur over multiple SW-R events thereby covering longer distances (268).”

“However, we now know that reactivations occur outside of sleep, i.e., in quiet wakefulness (Davidson et al., 2009; Diba and Buzsáki, 2007; Foster and Wilson, 2006; Karlsson and Frank, 2009; Kudrimoti et al., 1999), during initial learning (Singer et al., 2013), and during many states of cortical…”

“(Davidson et al., 2009; Wu and Foster, 2014), and the needed SPW-R clusters were prevented by the light stimulus.”

“…assembly replay during rest and sleep occurs in snippets and is faster, often compressed by at least a factor of ten compared to the behavioral timescale of neuronal activation (Davidson et al., 2009; Diba andBuzsáki, 2007; Euston et al., 2007; Foster and Wilson, 2006; Nádasdy et al., 1999).”

“, 2010) and animals (Louie and Wilson, 2001; Foster and Wilson, 2006; Karlsson and Frank, 2009; Davidson et al., 2009) have shown that hippocampal neuronal ensembles ‘‘replay’’ specific event representations following learning.”

“Examination of hippocampal activity during exploration of a larger environment revealed that both forward and reverse replay events can span long distances and extend across multiple SWR”

“Whereas previous studies (Wil- son and McNaughton, 1994; Jackson et al., 2006; O’Neill et al., 2006; Karlsson and Frank, 2009) have found increasing replay with experience, these studies did not compare tasks with different behavioral requirements in which parts of the environment were experienced more than others.”

“During replay sequences, place cells maintain their spatial relationships to each other (55, 109), resulting in trajectories through space occurring both in forward and reverse order (109, 110).”

“2010), and longer memories seem to be replayed across multiple sharp wave ripples (Davidson et al. 2009).”

“The presumed “reading out” in the SWR is accompanied by compression of the replay (Nadasdy et al. 1999, Euston et al. 2007, Ji & Wilson 2007); in other words, the postulated representation is played in “fast forward” (and, as noted below, under certain circumstances in “fast backward”).”

“The rest period was intended to be analogous to the awake resting periods in rodent studies in which hippocampal replay has been observed in spatial tasks (Davidson et al., 2009; Foster and Wilson, 2006; Karlsson and Frank, 2009).”

“In particular, reactivation during pauses in waking behavior frequently results in the sequential activity of place cells active on paths to or from the animal’s current location (Foster and Wilson, 2006; Diba and Buzsáki, 2007; Karlsson and Frank, 2009; Davidson et al., 2009).”

“First, a spatial Bayesian decoder (2), constructed from the firing-rate vectors of place cells (n = 491 cells) during track running (eight novel exploration sessions), was applied to all candidate ripple events (21) (figs.”

“This is particularly striking in the Davidson et al. (2009) study, in which replay can take 700 ms to wend its way around a bendy track without ever seeming to jump between physically close (but uncrossable) points, but this has been just as true in every experiment we are aware of.”

“Indeed, the typical duration of a replay event varies between 100 and 300ms [47,56,57], a 20 times increase in speed over the actual experience [58] — though replayed trajectories occurring during REM sleep progress at a more natural speed [59].”

“…et al., 2008; Crowe et al., 2010; Harvey et al., 2012), hippocampus (Na´dasdy et al., 1999; Louie and Wilson, 2001; Pastalkova et al., 2008; Davidson et al., 2009), basal ganglia (Barnes et al., 2005; Jin et al., 2009), cerebellum (Mauk and Buonomano, 2004), and area HVC of the songbird…”

“Finally, to corroborate the results obtained from the rank-order analysis of spike sequences, we applied a Bayesian spatial reconstruction algorithm (Davidson et al., 2009; Bendor and Wilson, 2012) to the data from the two rest sessions.”

“They noted that “the real importance of mental time travel applies to travel into the future rather than the past; that is, we predominantly stand in the present facing the future rather than looking back to the past.””

“, 2003; Ibos and Freedman, 2017), and spatial locations based on activity in the hippocampus (Zhang et al., 1998; Davidson et al., 2009).”

“…predicts the sequential activation of bands of grid cells that share a common phase on a specific 1D axis during route planning, analogous to place cell replay and preplay events (Foster and Wilson, 2006; Davidson et al., 2009; Pfeiffer and Foster, 2013; see S.G. Trettel and L.L. Colgin, 2014, Soc.”

“At the same time, place cells exhibit sequential activation of previous trajectories during activity replay [208, 219, 220, 221], which is hypothesized to be generated by recurrent connections in CA3, suggesting that recurrent and feedforward dynamics collaborate in the generation of place cell states; recent models a…”

“Does this mean that a fixed fraction of cells will have place fields regardless of the size of the environment?While fewer cells may express place fields in smaller and/or simpler mazes (Thompson and Best, 1989), in larger environments the number of place cells appears to be limited, with additional spatial coverage a…”

“In addition, ripple events can occur in groups (ripple bursts), in which ensembles of place cells represent extended exploratory behavior (Davidson et al. 2009).”

“Work in rodents exploring the link between awake SWRs and putative memory retrieval behaviors has demonstrated that sequences of hippocampal place cell assemblies, representing spatial and contextual information related to past experiences, are briefly replayed in the time window of the SWR (32, 34, 35, 41, 57).”

“These reactivation events frequently replay entire behavioral trajectories representing either past or possible future locations (Foster and Wilson, 2006; Diba and Buzsáki, 2007; Davidson et al., 2009; Karlsson and Frank, 2009; Gupta et al., 2010) and reactivation strength during and after an experience correlates wit…”

“Similarly, it remains unknown how multiple neural subsequences of varying length (Tanji, 2001; Wang et al., 2008; Davidson et al., 2009) are formed.”

“…a goodness-of-fit measure of the relationship between the temporal order of firing in the SWR and the associated decoded position in space, and a p value that assesses the likelihood of observing the R 2 value given the observed spikes (Figure S6) (Davidson et al., 2009; Karlsson and Frank, 2009).”

“These bursts often ''replay'' sequences from behavior (Kudrimoti et al., 1999; Foster and Wilson, 2006; Diba and Buzsá ki, 2007; Davidson et al., 2009; Karlsson and Frank, 2009; Gupta et al., 2010; Pfeiffer and Foster, 2013), suggesting that these sequences could support awake memory processes…”

“We identified these as units with a firing rate of less than 5 Hz (8, 62, 63) and with a “wide” (>0.”

“The best linear fit was calculated for each event (12) as the preplay score corresponding to the mean estimated likelihood that the animal was on the preplayed trajectory.”

“47 cm) was comparable to previous reports (Ambrose et al., 2016; Davidson et al., 2009; Farooq and Dragoi, 2019; O’Neill et al., 2017).”

“4a,b), as in previous studies 16,17,34 .”

“To test if fitted trajectories could be expected by chance alignment of position estimates, the replay score for each candidate event was compared to replay score distributions derived with two shuffles of the data, using the approach described in (Davidson et al., 2009).”

“200 Neuron 91, 194–204, July 6, 2016 rent location of the animal (Diba and Buzsáki, 2007; Foster and Wilson, 2006) but also encode trajectories that do not include the current location of the animal (Davidson et al., 2009; Gupta et al., 2010; Karlsson and Frank, 2009).”

“Directional place fields were calculated and were used to compute joint posterior probabilities over both position and direction (Davidson et al., 2009), for the already identified replays.”

“We ignore place cell firing that occurs because of the internal dynamics of the hippocampal network (such as reactivation of past experience [23]).”

“We also investigated the theta sequences of cell assemblies in the trials using a Bayesian decoding technique (Zhang et al., 1998; Davidson et al., 2009; Feng et al., 2015).”

“Of note, the duration of the replayed spike sequence in those experiments is also determined by the duration of the sharp wave ripple rather than reflecting the highly irregular behavior of the rat on the running track, suggesting that the sequence rather than time course of the experience is reactivate”

“To control for population differences in instantaneous place representation not captured by individual place fields, we performed Bayesian position decoding during running behavior 24 .”

“…signature of coherent, temporally structured population-level events within the hippocampus in which the pattern of neural activity often appears to encode behaviorally relevant information on a compressed timescale (Davidson et al., 2009; Foster & Wilson, 2006; Lee & Wilson, 2002) (Figure F11).”

“Wild rats have home ranges typically up to 50 m across, sometimes extending to hundreds of meters (11), and in larger laboratory environments (up to 18 m long or 3 m(2)) more neurons are recruited to be place cells, many cells have multiple fields, and place fields are wider (12-14).”

“A memoryless probability-based decoding algorithm was used to estimate the animal’s position based on the directional place fields and the spike trains (Davidson et al., 2009; Pfeiffer and Foster, 2013).”

“What exactly is being planned in replay episodes? Intriguingly, trajectory distance correlates strongly with the total duration of its replay (with concatenated ripple events for long trajectories) [175,176].”

“Neuronal reinstatement relating to consolidation has been described in rodents (Davidson et al., 2009; Karlsson and Frank, 2009; Foster and Wilson, 2006) and Received May 7, 2015; revised Aug. 20, 2015; accepted Aug. 31, 2015.”

“This is consistent with the range of compressions previously reported in animal models as well as in humans (Davidson et al., 2009; Liu et al., 2019), and it is too fast to be explained by mental rehearsal or motor imagery processes, which share a veridical duration with executed behavior (in this case, 1,000 to 2,500…”

“For each sequence, we circularly shifted the probability distribution of estimated positions by a random distance at each time bin 1,000 times (the same as the ''column-cycle'' shuffle described in Davidson et al., 2009 for sharp wave-associated sequences).”

“9 Discussion Place cell firing sequences are reactivated at compressed timescales during hippocampal SWRs (Nadasdy et al., 1999; Lee and Wilson, 2002; Foster and Wilson, 2006; Diba and Buzsaki, 2007; Davidson et al., 2009; Karlsson and Frank, 2009), generating conditions compatible with induction of STDP (Bi and Poo,…”

“With around 100 recorded single units from dorsal CA1 in each mouse (Figure S3C, 103±14 single units/mouse per 12 tetrode microdrive), we were able to estimate the animal’s position using the Bayesian decoding method (Davidson et al., 2009; Carr et al., 2011; Pfeiffer and Foster, 2013) during the RUN session in both C…”

“These sequences are replayed during subsequent sleep and quiet wakefulness, which are proposed to consolidate this memory (Skaggs and McNaughton, 1996; Davidson et al., 2009).”

“Extant evidence from animal studies suggests a role for SWRs in the reactivation of mnemonic information (Davidson et al., 2009; Karlsson and Frank, 2009; Peyrache et al., 2009; Shin et al., 2019; Takahashi, 2015; Wu et al., 2017) and offline consolidation of rodents’ spatial (Ego-Stengel and Wilson, 2010; Fernández-R…”

“…of possible future options (Pfeiffer and Foster, 2013; Singer et al., 2013), but SWR sequences also encode paths unrelated to immediate options (Davidson et al., 2009; Gupta et al., 2010) or even to the immediate task at hand (Jackson et al., 2006; Karlsson and Frank, 2009), and the prevalence…”

“However, another possibility is that the observed temporal compression complements previous studies suggesting that retrieval reactivates internal representations of an experience on a faster timescale than the original experience (33, 34).”

“Robustness of mPFC Trajectory Replay Events In addition to the decoding probability and smoothness scores, we used further measures to detect significant trajectory replay events (Davidson et al., 2009; Gupta et al., 2010).”