Human inference in changing environments with temporal structure.

“A standard method to fit and compare models is to maximize the log-likelihood of each model and compute its Bayesian Information Criterion (BIC), which includes a penalty as a function of the number of parameters in the model [56].”

“Bonferroni-Holm correction [44] is applied in panel B.”

“Monte Carlo algorithms perform such approximations by way of randomly sampling the spatial distribution; sequential Monte Carlo methods, or ‘particle filters’, were developed in the 1990s to address Hidden Markov Models, a class of hidden-state problems within which falls our inference task [50, 51, 52].”

“Monte Carlo algorithms perform such approximations by way of randomly sampling the spatial distribution; sequential Monte Carlo methods, or ‘particle filters’, were developed in the 1990s to address Hidden Markov Models, a class of hidden-state problems within which falls our inference task [50, 51, 52].”

“The Bayesian paradigm has been viewed as an extension of logic that enables reasoning with propositions whose truth or falsity is uncertain [59, 60].”

“In cognitive science, it has successfully accounted for a wide range of observations, including cue combination in humans [5, 4, 1, 61, 3, 6, 7, 2], sensorimotor control [8, 9, 62], integration of temporal statistics [37, 36, 10, 11, 38], perceptual multistability [63, 41, 42], and various aspects of cognition [64, 65…”

“5 Alternative models of response selection: ‘rational inattention’ and ad hoc repetition probability Aside from posterior sampling and motor noise, the so-called ‘rational inattention’ approach which has been gaining grounds in economics [79, 80, 81], suggests a different account of the variability of responses in dec…”

“In explore-exploit problems, ‘Thompson sampling’ [83] refers to a strategy in which one ‘explores’ by randomly choosing an action with the probability that this action maximizes the reward (instead of deterministically choosing the action most likely to maximize the reward).”

“Decision by posterior sampling, or ‘probability matching’, has been suggested by other decision-making experiments [74, 75, 76] and, more recently, perceptual experiments [77, 2, 41].”

“Unless otherwise stated, we focus our analyses on cases in which the surprise, xt+1 − ŝt, is in the [8,18] window, in which there is appreciable ambiguity in the signal.”

“The Bayesian paradigm has been viewed as an extension of logic that enables reasoning with propositions whose truth or falsity is uncertain [59, 60].”

“Our observations confirm and extend the results reported in the change-point literature on human inference in the context of Poisson statistics, by exploring a more ecological [27, 28, 29, 30, 31, 32, 33, 34, 35], non-Poisson, temporally structured environment.”

“Average learning rates in HI (blue) and HD (orange) conditions, at short run-lengths (τ̃ ∈ [5, 6]) and long run-lengths (τ̃ ∈ [9, 10]).”

“Our observations confirm and extend the results reported in the change-point literature on human inference in the context of Poisson statistics, by exploring a more ecological [27, 28, 29, 30, 31, 32, 33, 34, 35], non-Poisson, temporally structured environment.”

“In a number of experiments [36, 37, 38, 10, 11], a first event (a sensory cue, or a motor action such as a lever press) is followed by a second event, such as the delivery of a reward, or a ‘go’ signal triggering the next behavior.”

“In cognitive science, it has successfully accounted for a wide range of observations, including cue combination in humans [5, 4, 1, 61, 3, 6, 7, 2], sensorimotor control [8, 9, 62], integration of temporal statistics [37, 36, 10, 11, 38], perceptual multistability [63, 41, 42], and various aspects of cognition [64, 65…”

“Unless otherwise stated, we focus our analyses on cases in which the surprise, xt+1 − ŝt, is in the [8,18] window, in which there is appreciable ambiguity in the signal.”

“This suggests that humans use approximations when carrying out inference and estimation tasks [71, 72, 73].”

“At the implementation level, sample-based representations are well suited to learning in neural networks [97].”

“In cognitive science, it has successfully accounted for a wide range of observations, including cue combination in humans [5, 4, 1, 61, 3, 6, 7, 2], sensorimotor control [8, 9, 62], integration of temporal statistics [37, 36, 10, 11, 38], perceptual multistability [63, 41, 42], and various aspects of cognition [64, 65…”

“Although close to optimal in some specific paradigms [78], sampling is suboptimal in the context of our behavioral task.”

“A growing theoretical and experimental literature examines inference problems for this class of signals [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26].”

“In a number of experiments [36, 37, 38, 10, 11], a first event (a sensory cue, or a motor action such as a lever press) is followed by a second event, such as the delivery of a reward, or a ‘go’ signal triggering the next behavior.”

“The literature on Bayesian online inference in cognition, where belief is updated iteratively as a function of incoming information, is growing; examples can be found in word segmentation [68], sentence processing [69], conditioning [70], as well as in the change-point literature [16, 17, 15, 21, 22, 23, 25, 26].”

“The success of particle filters, also known as Sequential Monte Carlo method, in accounting for human behavior in an online inference task adds to a growing literature on sample-based representations in cognitive processes [40, 41, 42, 43].”

“The success of particle filters, also known as Sequential Monte Carlo method, in accounting for human behavior in an online inference task adds to a growing literature on sample-based representations in cognitive processes [40, 41, 42, 43].”

“Both these inference strategies reduce the memory load in the inference problem by stochastically trimming the posterior, in a fashion akin to the ‘pruning’ model proposed by [82] in the context of a decision-tree task.”

“In cognitive science, it has successfully accounted for a wide range of observations, including cue combination in humans [5, 4, 1, 61, 3, 6, 7, 2], sensorimotor control [8, 9, 62], integration of temporal statistics [37, 36, 10, 11, 38], perceptual multistability [63, 41, 42], and various aspects of cognition [64, 65…”

“Average learning rates in HI (blue) and HD (orange) conditions, at short run-lengths (τ̃ ∈ [5, 6]) and long run-lengths (τ̃ ∈ [9, 10]).”

“Here, the variability in neural activity can be interpreted in terms of sampling-based representations of probability [98, 97, 99, 42], and a number of neural network models performing probability sampling have been proposed [100, 41, 101, 102, 103].”

“All these studies, with the exception of the work of [13], focus on the history-independent case of random change points that obey Poisson temporal statistics.”

“In a similar spirit, particle filters have been shown to account for order effects in category learning [92] and observations about online sentence comprehension (such as the processing of ‘garden-path sentences’ [69]).”

“5 Alternative models of response selection: ‘rational inattention’ and ad hoc repetition probability Aside from posterior sampling and motor noise, the so-called ‘rational inattention’ approach which has been gaining grounds in economics [79, 80, 81], suggests a different account of the variability of responses in dec…”

“Average learning rates in HI (blue) and HD (orange) conditions, at short run-lengths (τ̃ ∈ [5, 6]) and long run-lengths (τ̃ ∈ [9, 10]).”

“Moreover, they account for a range of cognitive biases in the laboratory, such as base-rate neglect, conjunction fallacy, and the unpacking effect, as well as for human performance in complex, real-world tasks, and specific observations such as response variability and autocorrelation in perception and reasoning tasks…”

“In a number of experiments [36, 37, 38, 10, 11], a first event (a sensory cue, or a motor action such as a lever press) is followed by a second event, such as the delivery of a reward, or a ‘go’ signal triggering the next behavior.”

“In cognitive science, it has successfully accounted for a wide range of observations, including cue combination in humans [5, 4, 1, 61, 3, 6, 7, 2], sensorimotor control [8, 9, 62], integration of temporal statistics [37, 36, 10, 11, 38], perceptual multistability [63, 41, 42], and various aspects of cognition [64, 65…”

“Several studies have reported that the responses of human subjects in explore-exploit tasks appeared consistent with Thompson sampling [84, 85, 86].”

“Decision by posterior sampling, or ‘probability matching’, has been suggested by other decision-making experiments [74, 75, 76] and, more recently, perceptual experiments [77, 2, 41].”

“In cognitive science, it has successfully accounted for a wide range of observations, including cue combination in humans [5, 4, 1, 61, 3, 6, 7, 2], sensorimotor control [8, 9, 62], integration of temporal statistics [37, 36, 10, 11, 38], perceptual multistability [63, 41, 42], and various aspects of cognition [64, 65…”

“The literature on Bayesian online inference in cognition, where belief is updated iteratively as a function of incoming information, is growing; examples can be found in word segmentation [68], sentence processing [69], conditioning [70], as well as in the change-point literature [16, 17, 15, 21, 22, 23, 25, 26].”

“Here, the variability in neural activity can be interpreted in terms of sampling-based representations of probability [98, 97, 99, 42], and a number of neural network models performing probability sampling have been proposed [100, 41, 101, 102, 103].”

“Aside from stochasticity in the inference step and in the response-selection step, noise in the sensory observation is a possible alternate account of behavioral variability, discussed, among others, by [90] and [58].”

“Several studies have reported that the responses of human subjects in explore-exploit tasks appeared consistent with Thompson sampling [84, 85, 86].”

“Our observations confirm and extend the results reported in the change-point literature on human inference in the context of Poisson statistics, by exploring a more ecological [27, 28, 29, 30, 31, 32, 33, 34, 35], non-Poisson, temporally structured environment.”

“[27], [28, 29], and [30] have recorded the motor activity of both rodents and human subjects over the course of several days.”

“The literature on Bayesian online inference in cognition, where belief is updated iteratively as a function of incoming information, is growing; examples can be found in word segmentation [68], sentence processing [69], conditioning [70], as well as in the change-point literature [16, 17, 15, 21, 22, 23, 25, 26].”

“Here, the variability in neural activity can be interpreted in terms of sampling-based representations of probability [98, 97, 99, 42], and a number of neural network models performing probability sampling have been proposed [100, 41, 101, 102, 103].”

“In addition, [95] consider only the case of a single particle.”

“[53] argue that, for some forms of random perturbations of a posterior probability density, maximizing the randomly perturbed function yields similar results to sampling from a modified posterior density function obtained as a power of the correct posterior: pκ(s) ∝ p(s|x1:t).”

“) Previous uses of particle-filter methods in the context of a variety of cognitive tasks yielded best-fitting numbers of particles which ranged from one to several hundreds: from one to 400 particles, with a mean of 56, in [14]; 130 particles (but 70 when subjects simultaneously perform a distractor task) in [94]; ar…”

“Average learning rates in HI (blue) and HD (orange) conditions, at short run-lengths (τ̃ ∈ [5, 6]) and long run-lengths (τ̃ ∈ [9, 10]).”

“Another possible explanation is that even though the new estimate falls on a nearby location, a motor cost prohibits a move if it is not sufficiently extended to be ‘worth it’ [45, 46, 47, 48].”

“Another possible explanation is that even though the new estimate falls on a nearby location, a motor cost prohibits a move if it is not sufficiently extended to be ‘worth it’ [45, 46, 47, 48].”

“In cognitive science, it has successfully accounted for a wide range of observations, including cue combination in humans [5, 4, 1, 61, 3, 6, 7, 2], sensorimotor control [8, 9, 62], integration of temporal statistics [37, 36, 10, 11, 38], perceptual multistability [63, 41, 42], and various aspects of cognition [64, 65…”

“In cognitive science, it has successfully accounted for a wide range of observations, including cue combination in humans [5, 4, 1, 61, 3, 6, 7, 2], sensorimotor control [8, 9, 62], integration of temporal statistics [37, 36, 10, 11, 38], perceptual multistability [63, 41, 42], and various aspects of cognition [64, 65…”

“The literature on Bayesian online inference in cognition, where belief is updated iteratively as a function of incoming information, is growing; examples can be found in word segmentation [68], sentence processing [69], conditioning [70], as well as in the change-point literature [16, 17, 15, 21, 22, 23, 25, 26].”

“[27], [28, 29], and [30] have recorded the motor activity of both rodents and human subjects over the course of several days.”

“Decision by posterior sampling, or ‘probability matching’, has been suggested by other decision-making experiments [74, 75, 76] and, more recently, perceptual experiments [77, 2, 41].”

“Likewise, our results come to complement those of a number of studies on perception and decision-making that also investigate inference from stimuli with temporal statistics [36, 37, 38, 10, 11, 39].”

“) Previous uses of particle-filter methods in the context of a variety of cognitive tasks yielded best-fitting numbers of particles which ranged from one to several hundreds: from one to 400 particles, with a mean of 56, in [14]; 130 particles (but 70 when subjects simultaneously perform a distractor task) in [94]; ar…”

“The literature on Bayesian online inference in cognition, where belief is updated iteratively as a function of incoming information, is growing; examples can be found in word segmentation [68], sentence processing [69], conditioning [70], as well as in the change-point literature [16, 17, 15, 21, 22, 23, 25, 26].”

“The literature on Bayesian online inference in cognition, where belief is updated iteratively as a function of incoming information, is growing; examples can be found in word segmentation [68], sentence processing [69], conditioning [70], as well as in the change-point literature [16, 17, 15, 21, 22, 23, 25, 26].”

“Here, the variability in neural activity can be interpreted in terms of sampling-based representations of probability [98, 97, 99, 42], and a number of neural network models performing probability sampling have been proposed [100, 41, 101, 102, 103].”

“The literature on Bayesian online inference in cognition, where belief is updated iteratively as a function of incoming information, is growing; examples can be found in word segmentation [68], sentence processing [69], conditioning [70], as well as in the change-point literature [16, 17, 15, 21, 22, 23, 25, 26].”

“In cognitive science, it has successfully accounted for a wide range of observations, including cue combination in humans [5, 4, 1, 61, 3, 6, 7, 2], sensorimotor control [8, 9, 62], integration of temporal statistics [37, 36, 10, 11, 38], perceptual multistability [63, 41, 42], and various aspects of cognition [64, 65…”

“Another interpretation of this sampling strategy is proposed by [2]: in the case of an integer κ and a Gaussian posterior, the mean of κ samples drawn from the posterior is a Gaussian random variable, with a standard deviation equal to that of the posterior scaled by 1/ √ κ; i.”

“The literature on Bayesian online inference in cognition, where belief is updated iteratively as a function of incoming information, is growing; examples can be found in word segmentation [68], sentence processing [69], conditioning [70], as well as in the change-point literature [16, 17, 15, 21, 22, 23, 25, 26].”

“[27], [28, 29], and [30] have recorded the motor activity of both rodents and human subjects over the course of several days.”

“tion, etc); they have also been put forth as candidate cognitive algorithms [96, 72].”

“A similar picture has been advanced in the context of prediction tasks, where ‘over-reaction’ – effectively a biased, enhanced learning rate – results from the compression of information stored in memory [87, 88, 89].”

“Our observations confirm and extend the results reported in the change-point literature on human inference in the context of Poisson statistics, by exploring a more ecological [27, 28, 29, 30, 31, 32, 33, 34, 35], non-Poisson, temporally structured environment.”

“Likewise, our results come to complement those of a number of studies on perception and decision-making that also investigate inference from stimuli with temporal statistics [36, 37, 38, 10, 11, 39].”

“A growing theoretical and experimental literature examines inference problems for this class of signals [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26].”

“) Previous uses of particle-filter methods in the context of a variety of cognitive tasks yielded best-fitting numbers of particles which ranged from one to several hundreds: from one to 400 particles, with a mean of 56, in [14]; 130 particles (but 70 when subjects simultaneously perform a distractor task) in [94]; ar…”

“Here, we mention that, even though models with temporal correlation such as the particle filter have been used to capture cognitive processes, to the best of our knowledge Bayesian model selection using the BIC has not been applied to them, except in the case of a binary categorization task [57].”

“Sampling from the modified posterior yields a behavior that interpolates between posterior sampling (for κ = 1) and maximizing (for κ→∞); it yields a family of softmax operations over the posterior [54, 55].”

“[27], [28, 29], and [30] have recorded the motor activity of both rodents and human subjects over the course of several days.”

“A similar picture has been advanced in the context of prediction tasks, where ‘over-reaction’ – effectively a biased, enhanced learning rate – results from the compression of information stored in memory [87, 88, 89].”

“Decision by posterior sampling, or ‘probability matching’, has been suggested by other decision-making experiments [74, 75, 76] and, more recently, perceptual experiments [77, 2, 41].”

“A growing theoretical and experimental literature examines inference problems for this class of signals [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26].”

“A growing theoretical and experimental literature examines inference problems for this class of signals [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26].”

“Random errors in a calculation that are subsequently incorporated into later calculations are hallmarks of multistage sampling processes, such as ancestral sampling or sequential Monte Carlo (Daw & Courville, 2008; Prat-Carrabin et al., 2021; Sanborn et al., 2010).”

“Some experimental tasks are indeed set in such a context, as when human subjects are asked to make inferences in volatile environments [16, 19, 21, 28, 29] (see also models of temporal discounting [30–34]).”

“Current understanding of how latent uncertainty factors drive trial-by-trial changes in apparent learning rates have been largely informed by the utilisation of predictive inference tasks that employ magnitude-based quantities, such as spatial positions [17,19,20] or numerical values [18].”