The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
Homeostatic reinforcement learning for integrating reward collection and physiological stability
Open accessPublished version of a preprint
Abstract
Efficient regulation of internal homeostasis and defending it against perturbations requires adaptive behavioral strategies. However, the computational principles mediating the interaction between homeostatic and associative learning processes remain undefined. Here we use a definition of primary rewards, as outcomes fulfilling physiological needs, to build a normative theory showing how learning motivated behaviors may be modulated by internal states. Within this framework, we mathematically prove that seeking rewards is equivalent to the fundamental objective of physiological stability, defining the notion of physiological rationality of behavior. We further suggest a formal basis for temporal discounting of rewards by showing that discounting motivates animals to follow the shortest path in the space of physiological variables toward the desired setpoint. We also explain how animals learn to act predictively to preclude prospective homeostatic challenges, and several other behavioral patterns. Finally, we suggest a computational role for interaction between hypothalamus and the brain reward system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current models of decision-making do not incorporate all of these aspects, and first attempts of accounting for homeostasis and allostasis in formal models of decision-making have only surfaced relatively recently (e.g., Keramati and Gutkin, 2014; Penny and Stephan, 2014; Pezzulo et al., 2015).”
From Allostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression (2016) · cites this paper as Keramati and Gutkin, 2014
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current models of decision-making do not incorporate all of these aspects, and first attempts of accounting for homeostasis and allostasis in formal models of decision-making have only surfaced relatively recently (e.g., Keramati and Gutkin, 2014; Penny and Stephan, 2014; Pezzulo et al., 2015).”
From Allostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression (2016) · cites this paper as Keramati and Gutkin, 2014
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Prospect theory: An analysis of decision under risk
Kahneman, D. J., & Tversky, A. (1988) · Cambridge University Press eBooks · doi:10.1017/cbo9780511609220.014 · Cited by 32,720
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 3×
“A self-organizing system (i.e. an organism) can be defined as a system that opposes the second law of thermodynamics (Friston, 2010).”
BackgroundMethodInfluentialSakurai, T., Amemiya, A., Ishii, M., Matsuzaki, I., Chemelli, R. M., Tanaka, H., Williams, S. C., Richardson, J. A., Kozlowski, G. P., Wilson, S., et al. (1998) · Cell · doi:10.1016/s0092-8674(00)80949-6 · Cited by 5,501
Skinner, B. F. (1938) · Cited by 3,571
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184
Flagged first, then by citations
Research works printed but not linked: 3
- Bernard, 1957, Lectures on the physiological properties and the pathological alternations of the liquids of the organism: Third lecture, 89
- Marr, 1982, Vision
- Sibly, 1974, State Space Approach to Motivation, Motivational Control System Analysis
Cited by
Sorted by citations
Items 101–102 of 102
Bertrand, M., Chabardès, S., Fontanier, V., Procyk, E., Bastin, J., & Piallat, B. (2024) · Frontiers in Neuroscience · doi:10.3389/fnins.2024.1257579 · Cited by 7
Fundamental features of social environments determine rate of social affiliation
Garud, S., Rascu, M., Hamilton, S., Yu, I., Rushworth, M. F. S., & Klein-Flügge, M. C. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2506243122 · Cited by 2
Metadata
- Authors
- Keramati, M., & Gutkin, B.
- Affiliations
- Inserm (FR)
- National Research University Higher School of Economics (RU)
- Corresponding author
- Keramati, Mehdi
- Gutkin, Boris
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2014-12-02
- Volume
- 3
- ISSN
- 2050-084X (ISSN-L)
- PMC
- PMC4270100
- OpenAlex
- W2142110758
- Cited by
- 218
- References
- 80
- Citations by year
- 2026: 23 · 2025: 35 · 2024: 29 · 2023: 15 · 2022: 37 · 2021: 23 · 2020: 16 · 2019: 16 · 2018: 10 · 2017: 8 · 2016: 5 · 2015: 1
- FWCI
- 2.13
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- eLife (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- City Research Online (City University London) (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Gatsby Charitable Foundation
- National Research University Higher School of Economics (Basic Research Program)
- Institut national de la santé et de la recherche médicale (INSERM U960, France)
- Center for Research and Interdisciplinary (Frontiers du Vivant)
- Agence Nationale de la Recherche (ANR-10-LABX-0087 IEC, France, ANR-10-IDEX-0001-02 PSL, France)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- physiological stability, primary reward, normative theory, delay discounting, predictive learning, hypothalamus, brain reward system, adaptive behavior
- MeSH terms
- Animals, Behavior, Computer Simulation, Conditioning, Psychological, Extinction, Psychological, Homeostasis, Learning, Models, Neurological, Reinforcement, Psychology, Reward, Satiety Response
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
