CoChat
Sign Up
Cited by 45Decision-Making and Behavioral EconomicsNeural and Behavioral Psychology StudiesBayesian Modeling and Causal Inference

Computational processes of simultaneous learning of stochasticity and volatility in humans

Open access
Piray, Payam, Daw, Nathaniel D.·2024·Nature Communications
DOI 10.1038/s41467-024-53459-zPMC PMC11494056
Open this paper in CoChat — free accountOpen PDF

Abstract

Adapting to uncertain environments is crucial for survival. This study explores computational challenges in distinguishing two types of noise: volatility and stochasticity. Volatility refers to diffusion noise in latent causes, requiring a higher learning rate, while stochasticity introduces moment-to-moment observation noise and reduces learning rate. For the learner, dissociating their effects on one’s observations is challenging because they both increase the variance of observations. Previous research examined these factors separately, but it remains unclear whether and how humans dissociate them. In two large-scale experiments, through novel behavioral tasks and computational modeling, we report compelling evidence of humans dissociating volatility and stochasticity solely based on their observations. We observed contrasting effects of volatility and stochasticity on learning rates, consistent with statistical principles. These results are consistent with a computational model that estimates volatility and stochasticity by balancing their dueling effects, but not with a number of other models that fail to make this distinction. This research elucidates computational processes behind adaptive learning in uncertain environments.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-024-53459-z

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Human participants are able to detect and distinguish dynamic changes in both volatility and observation noise at the same time.

    “and recent empirical work has demonstrated that human participants are indeed able to do so (Piray & Daw, 2024; Foucault et al., 2025).”

    From The generalized Hierarchical Gaussian Filter (2026) · cites this paper as Piray & Daw, 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 58

  1. [2]Dayan, P., Kakade, S. & Montague, P. R. Learning and selective attention. Nat. Neurosci. 3, 1218–1223 (2000). 11127841 · DOI · Paper page
  2. [3]Courville, A. C., Daw, N. D. & Touretzky, D. S. Bayesian theories of conditioning in a changing world. Trends Cognit. Sci. 10, 294–300 (2006). 16793323 · DOI · Paper page
  3. [4]Daunizeau, J. et al. Observing the observer (I): meta-bayesian models of learning and decision-making. PLoS ONE 5, e15554 (2010). 21179480 PMC3001878 · DOI · Paper page
  4. [7]Nassar, M. R. et al. Rational regulation of learning dynamics by pupil-linked arousal systems. Nat. Neurosci. 15, 1040–1046 (2012). 22660479 PMC3386464 · DOI · Paper page
  5. [8]Iglesias, S. et al. Hierarchical prediction errors in midbrain and basal forebrain during sensory learning. Neuron 80, 519–530 (2013). 24139048 · DOI · Paper page
  6. [9]McGuire, J. T., Nassar, M. R., Gold, J. I. & Kable, J. W. Functionally dissociable influences on learning rate in a dynamic environment. Neuron 84, 870–881 (2014). 25459409 PMC4437663 · DOI · Paper page
  7. [11]Mathys, C., Daunizeau, J., Friston, K. J. & Stephan, K. E. A bayesian foundation for individual learning under uncertainty. Front. Hum. Neurosci. 5, 39 (2011). 21629826 PMC3096853 · DOI · Paper page
  8. [12]Browning, M., Behrens, T. E., Jocham, G., O’Reilly, J. X. & Bishop, S. J. Anxious individuals have difficulty learning the causal statistics of aversive environments. Nat. Neurosci. 18, 590–596 (2015). 25730669 PMC4644067 · DOI · Paper page
  9. [13]Piray, P. & Daw, N. D. A model for learning based on the joint estimation of stochasticity and volatility. Nat. Commun. 12, 6587 (2021). 34782597 PMC8592992 · DOI · Paper page
  10. [14]Piray, P. & Daw, N. D. A simple model for learning in volatile environments. PLoS Comput. Biol. 16, e1007963 (2020). 32609755 PMC7329063 · DOI
  11. [16]Brazil, I. A., Mathys, C. D., Popma, A., Hoppenbrouwers, S. S. & Cohn, M. D. Representational uncertainty in the brain during threat conditioning and the link with psychopathic traits. Biol. Psychiatry Cognit. Neurosci. Neuroimaging 2, 689–695 (2017). 29560903 · DOI
  12. [17]Cole, D. M. et al. Atypical processing of uncertainty in individuals at risk for psychosis. NeuroImage: Clin. 26, 102239 (2020). 32182575 PMC7076146 · DOI
  13. [18]Deserno, L. et al. Volatility estimates increase choice switching and relate to prefrontal activity in Schizophrenia. Biol. Psychiatry. Cognit. Neurosci. Neuroimaging 5, 173–183 (2020). 31937449 · DOI
  14. [19]Diaconescu, A. O., Wellstein, K. V., Kasper, L., Mathys, C. & Stephan, K. E. Hierarchical Bayesian models of social inference for probing persecutory delusional ideation. J. Abnorm. Psychol. 129, 556–569 (2020). 32757600 · DOI
  15. [20]Farashahi, S. et al. Metaplasticity as a NEURAL SUBSTRATE FOR ADAPTIVE LEARNING AND CHOICE UNDER UNCERTAINty. Neuron 94, 401–414.e6 (2017). 28426971 PMC5515734 · DOI
  16. [21]Gagne, C., Dayan, P. & Bishop, S. J. When planning to survive goes wrong: predicting the future and replaying the past in anxiety and PTSD. Curr. Opin. Behav. Sci. 24, 89–95 (2018). · DOI
  17. [22]Katthagen, T. et al. Modeling subjective relevance in schizophrenia and its relation to aberrant salience. PLoS Comput. Biol. 14, e1006319 (2018). 30096179 PMC6105009 · DOI · Paper page
  18. [24]Paliwal, S. et al. Subjective estimates of uncertainty during gambling and impulsivity after subthalamic deep brain stimulation for Parkinson’s disease. Sci. Rep. 9, 14795 (2019). 31616015 PMC6794275 · DOI
  19. [25]Piray, P., Ly, V., Roelofs, K., Cools, R. & Toni, I. Emotionally aversive cues suppress neural systems underlying optimal learning in socially anxious individuals. J. Neurosci. 39, 1445–1456 (2019). 30559152 PMC6381249 · DOI
  20. [26]Powers, A. R., Mathys, C. & Corlett, P. R. Pavlovian conditioning-induced hallucinations result from overweighting of perceptual priors. Science 357, 596–600 (2017). 28798131 PMC5802347 · DOI · Paper page
  21. [27]Diaconescu, A. O. et al. Inferring on the Intentions of Others by Hierarchical Bayesian Learning. PLoS Comput. Biol10, e1003810 (2014). PMC4154656 25187943
  22. [28]Reed, E. J. et al. Paranoia as a deficit in non-social belief updating. eLife9, e56345 (2020). PMC7326495 32452769
  23. [29]Nassar, M. R., Wilson, R. C., Heasly, B. & Gold, J. I. An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J. Neurosci. 30, 12366–12378 (2010). 20844132 PMC2945906 · DOI
  24. [30]Lee, S., Gold, J. I. & Kable, J. W. The human as delta-rule learner. PsycNET. Decis. 7, 55–66 (2020). · DOI
  25. [31]Mackintosh, N. J. A theory of attention: variations in the associability of stimuli with reinforcement. Psychol. Rev. 82, 276–298 (1975). · DOI · Paper page
  26. [33]Hall, G. & Pearce, J. M. Restoring the associability of a Pre-Exposed CS by a Surprising Event. Q. J. Exp. Psychol. Sect. B 34, 127–140 (1982). · DOI
  27. [34]Holland, P. C. & Gallagher, M. Amygdala circuitry in attentional and representational processes. Trends Cognit. Sci. 3, 65–73 (1999). 10234229 · DOI
  28. [35]Holland, P. C. & Schiffino, F. L. Mini-review: prediction errors, attention and associative learning. Neurobiol. Learn. Mem. 131, 207–215 (2016). 26948122 PMC4862921 · DOI
  29. [36]Costa, V. D., Dal Monte, O., Lucas, D. R., Murray, E. A. & Averbeck, B. B. Amygdala and ventral striatum make distinct contributions to reinforcement learning. Neuron 92, 505–517 (2016). 27720488 PMC5074688 · DOI
  30. [37]Piray, P., Dezfouli, A., Heskes, T., Frank, M. J. & Daw, N. D. Hierarchical Bayesian inference for concurrent model fitting and comparison for group studies. PLOS Comput. Biol. 15, e1007043 (2019). 31211783 PMC6581260 · DOI
  31. [38]Gibbon, J., Farrell, L., Locurto, C. M., Duncan, H. J. & Terrace, H. S. Partial reinforcement in autoshaping with pigeons. Anim. Learn. Behav. 8, 45–59 (1980). · DOI
  32. [39]Nassar, M. R. et al. Age differences in learning emerge from an insufficient representation of uncertainty in older adults. Nat Commun7, 11609 (2016). PMC4906358 27282467
  33. [40]Kalman, R. E. A new approach to linear filtering and prediction problems. Trans. ASME J. Basic Eng. 82, 35–45 (1960). · DOI · Paper page
  34. [41]Doucet, A., de Freitas, N., Murphy, K. P. & Russell, S. J. Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks. In Proc. 16th Conference on Uncertainty in Artificial Intelligence 176–183 (Morgan Kaufmann Publishers Inc., 2000).
  35. [42]Doucet, A. & Johansen, A. M. A tutorial on particle filtering and smoothing: Fifteen years later. in Handbook of Nonlinear Filtering (eds. Crisan, D. & Rozovskii, B.) (Oxford University Press, 2011).
  36. [43]Gelbart, M. A., Snoek, J. & Adams, R. P. Bayesian optimization with unknown constraints. in Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence 250259 (AUAI Press, Arlington, Virginia, USA, 2014)
  37. [44]Snoek, J., Larochelle, H. & Adams, R. P. Practical bayesian optimization of machine learning algorithms. 10.48550/arXiv.1206.2944 (2012). · DOI · Paper page
  38. [45]d’Acremont, M. & Bossaerts, P. Neural mechanisms behind identification of Leptokurtic Noise and Adaptive Behavioral Response. Cereb. Cortex 26, 1818–1830 (2016). 26850528 PMC4785960 · DOI
  39. [46]Sutton, R. S. Adapting bias by gradient descent: an incremental version of delta-bar-delta. In Proc. tenth national conference on Artificial intelligence 171–176 (AAAI Press, San Jose, 1992).
  40. [47]Findling, C., Skvortsova, V., Dromnelle, R., Palminteri, S. & Wyart, V. Computational noise in reward-guided learning drives behavioral variability in volatile environments. Nat. Neurosci. 22, 2066–2077 (2019). 31659343 · DOI
  41. [48]Diederen, K. M. J. & Schultz, W. Scaling prediction errors to reward variability benefits error-driven learning in humans. J. Neurophysiol. 114, 1628–1640 (2015). 26180123 PMC4563025 · DOI · Paper page
  42. [49]Hartley, C. A. & Phelps, E. A. Anxiety and decision-making. Biol. Psychiatry 72, 113–118 (2012). 22325982 PMC3864559 · DOI
  43. [52]Aylward, J. et al. Altered learning under uncertainty in unmedicated mood and anxiety disorders. Nat. Hum. Behav. 3, 1116–1123 (2019). 31209369 PMC6790140 · DOI
  44. [53]Huang, H., Thompson, W. & Paulus, M. P. Computational dysfunctions in anxiety: failure to differentiate signal from noise. Biol. Psychiatry 82, 440–446 (2017). 28838468 PMC5576575 · DOI
  45. [55]Khorsand, P. & Soltani, A. Optimal structure of metaplasticity for adaptive learning. PLoS Comput. Biol. 13, e1005630 (2017). 28658247 PMC5509349 · DOI
  46. [56]Pulcu, E. & Browning, M. Humans adapt rationally to approximate estimates of uncertainty. 2023.11.26.568699 Preprint at 10.1101/2023.11.26.568699 (2023). PMC12237418 40627672 · DOI
  47. [57]Andrieu, C., Doucet, A. & Holenstein, R. Particle Markov chain Monte Carlo methods. J. R. Stat. Soc. Ser. B 72, 269–342 (2010). · DOI
  48. [59]Lee, J. K., Rouault, M. & Wyart, V. Adaptive tuning of human learning and choice variability to unexpected uncertainty. Sci. Adv. 9, eadd0501 (2023). 36989365 PMC10058239 · DOI
  49. [60]FeldmanHall, O. & Nassar, M. R. The computational challenge of social learning. Trends Cognit. Sci. 25, 1045–1057 (2021). 34583876 PMC8585698 · DOI
  50. [61]Mahmoodi, A., Luo, S., Harbison, C., Piray, P. & Rushworth, M. F. S. Human hippocampus and dorsomedial prefrontal cortex infer and update latent causes during social interaction. Neuron 112, 1–14 (2024). 39353432
  51. [62]Wilson, R. C., Takahashi, Y. K., Schoenbaum, G. & Niv, Y. Orbitofrontal cortex as a cognitive map of task space. Neuron 81, 267–279 (2014). 24462094 PMC4001869 · DOI · Paper page
  52. [63]Niv, Y. Learning task-state representations. Nat. Neurosci. 22, 1544–1553 (2019). 31551597 PMC7241310 · DOI · Paper page
  53. [64]Gershman, S. J. A unifying probabilistic view of associative learning. PLoS Comput. Biol.11, e1004567 (2015). PMC4633133 26535896
  54. [65]Piray, P. & Daw, N. D. Linear reinforcement learning in planning, grid fields, and cognitive control. Nat. Commun. 12, 4942 (2021). 34400622 PMC8368103 · DOI
  55. [66]Hunt, L. T. et al. Formalizing planning and information search in naturalistic decision-making. Nat. Neurosci. 24, 1051–1064 (2021). 34155400 · DOI
  56. [67]Collins, A. G. E. & Cockburn, J. Beyond dichotomies in reinforcement learning. Nat. Rev. Neurosci. 21, 576–586 (2020). 32873936 PMC7800310 · DOI
  57. [69]Douc, R. & Cappe, O. Comparison of resampling schemes for particle filtering. In ISPA 2005. Proc. 4th International Symposium on Image and Signal Processing and Analysis, 2005. 64–69. 10.1109/ISPA.2005.195385 (2005). · DOI
  58. [70]Piray, P. Computational processes of simultaneous learning of stochasticity and volatility in humans: code and data. Zenodo 10.5281/zenodo.13840905 (2024). PMC11494056 39433765 · DOI

Cited by

Sorted by citations

Authors
Piray, P., & Daw, N. D.
Affiliations
  • University of Southern California (US)
  • Princeton University (US)
Corresponding author
Piray, Payam
ORCID
Venue
Nature Communications
Publisher
Springer Science and Business Media LLC
Published
2024-10-21
Volume
15
Issue
1
Pages
9073-9073
ISSN
2041-1723 (ISSN-L)
DOI
10.1038/s41467-024-53459-z
PMC
PMC11494056
OpenAlex
W4403604892
Cited by
45
References
9
Citations by year
2026: 19 · 2025: 22 · 2024: 4
FWCI
16.70
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
  • PsyArXiv (OSF Preprints) (submittedVersion)
Funders
  • National Science Foundation (1822571)
  • U.S. Department of Health and Human Services (R21MH134217)
  • National Institute of Mental Health (R21MH134217)
  • Army Research Office (W911NF-16-1-, W911NF, W911NF‑16‑1‑0474)
Topics
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
Keywords
volatility, randomness, learning rate, adaptive learning, prediction, uncertainty, diffusion noise, measurement noise, computational models of learning
MeSH terms
Adult, Computer Simulation, Decision Making, Female, Humans, Learning, Male, Stochastic Processes, Uncertainty, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review