Episodic retrieval for model-based evaluation in sequential decision tasks

Abstract

It has long been hypothesized that episodic memory supports adaptive decision making by enabling mental simulation of future events. Yet, memory research is often carried out in settings that are far removed from ecological contexts of decision making, and models of adaptive choice, conversely, only invoke episodic memory in highly stylized terms, if it all. To address these gaps, we propose a novel process-level model of choice that grounds model-based evaluation in empirically informed dynamics of episodic recall. In this model, the probability of retrieving each available memory sample is given by the Successor Representation (SR), a biologically plausible world model in reinforcement learning. The evolution of these probabilities based on past retrievals, in turn, is dictated by the Temporal Context Model (TCM), a prominent model of episodic retrieval. Through a series of simulations, we demonstrate that the patterns of episodic retrieval suggested by this model enables flexible computation of decision variables. On this basis, we argue that a number of previously described features of episodic memory serve an adaptive purpose in sequential decision making. For instance, we show that the classic retrieval bias known as contiguity effect, when viewed from a decision making perspective, leads to model-based rollouts for forward simulation. We also show that features of episodic memory such as emotional modulation enable generalization and efficient decisions given limited experience. By bridging theoretical models across these two domains, we make a set of theoretical predictions linking episodic memory properties to adaptive choice in sequential tasks that may guide future empirical endeavors.