The DOI matches its Crossref record; 106 of the 106 references it cites were found in a registry or an index.
Thermodynamics as a theory of decision-making with information-processing costs
Open accessAbstract
Perfectly rational decision-makers maximize expected utility, but crucially ignore the resource costs incurred when determining optimal actions. Here, we propose a thermodynamically inspired formalization of bounded rational decision-making where information processing is modelled as state changes in thermodynamic systems that can be quantified by differences in free energy. By optimizing a free energy, bounded rational decision-makers trade off expected utility gains and information-processing costs measured by the relative entropy. As a result, the bounded rational decision-making problem can be rephrased in terms of well-known variational principles from statistical physics. In the limit when computational costs are ignored, the maximum expected utility principle is recovered. We discuss links to existing decision-making frameworks and applications to human decision-making experiments that are at odds with expected utility theory. Since most of the mathematical machinery can be borrowed from statistical physics, the main contribution is to re-interpret the formalism of thermodynamic free-energy differences in terms of bounded rational decision-making and to discuss its relationship to human decision-making experiments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Prospect theory: An analysis of decision under risk
Kahneman, D., & Tversky, A. (1988) · Decision, Probability and Utility · doi:10.1017/cbo9780511609220.014 · Cited by 32,474
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
Theory of games and economic behavior
(2019) · 100 Years of Math Milestones · doi:10.1090/mbk/121/32 · Cited by 16,632
Advances in prospect theory: Cumulative representation of uncertainty
Tversky, A., & Kahneman, D. (1992) · Journal of Risk and Uncertainty · doi:10.1007/bf00122574 · Cited by 15,467
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Conditional logit analysis of qualitative choice behavior
McFadden, D. L. (1972) · eScholarship (California Digital Library) · Cited by 14,203
Cragg, J. G., & Zarembka, P. (1975) · The Canadian Journal of Economics · doi:10.2307/134075 · Cited by 8,342
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 2×
“A free energy principle based on the variational Bayes approach has recently also been proposed as a theoretical framework to understand brain function [82, 83].”
Background“Risk, Ambiguity, and the Savage Axioms.”
Ellsberg, D. (1961) · The Quarterly Journal of Economics · doi:10.2307/1884324 · Cited by 7,891
Discrete Choice Methods with Simulation
Train, K. E. (2001) · doi:10.1017/cbo9780511805271 · Cited by 7,784
(2001) · Empirical Direction in Design and Analysis · doi:10.4324/9781410600004-24 · Cited by 6,808
Kahneman, D., & Tversky, A. (1984) · American Psychologist · doi:10.1037/0003-066x.39.4.341 · Cited by 6,544
Information theory, inference, and learning algorithms
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5949 · Cited by 6,362
Rational choice and the structure of the environment.
Simon, H. A. (1956) · Psychological Review · doi:10.1037/h0042769 · Cited by 5,919
Gumbel, E. J. (1958) · Columbia University Press eBooks · doi:10.7312/gumb92958 · Cited by 5,567
Maps of Bounded Rationality: Psychology for Behavioral Economics
Kahneman, D. (2003) · American Economic Review · doi:10.1257/000282803322655392 · Cited by 5,377
Irreversibility and heat generation in the computing process
Landauer, R. (1961) · IBM Journal of Research and Development · doi:10.1147/rd.53.0183 · Cited by 4,980
Information Theory, Inference, and Learning Algorithms
(2004) · Kybernetes · doi:10.1108/03684920410534506 · Cited by 4,937
The Foundations of Statistics.
Good, I. J., & Savage, L. J. (1955) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2343133 · Cited by 4,582
Models of Bounded Rationality: Empirically Grounded Economic Reason
Simon, H. A. (1997) · The MIT Press eBooks · doi:10.7551/mitpress/4711.001.0001 · Cited by 4,205
Allais, M. (1953) · Econometrica · doi:10.2307/1907921 · Cited by 4,112
Theory of Games and Economic Behavior
Hawkins, D. (1945) · Philosophy of Science · doi:10.1086/286866 · Cited by 3,887
Behavioral Game Theory: Experiments in Strategic Interaction
Camerer, C. F. (2003) · Cited by 3,722
Logical Reversibility of Computation
Bennett, C. H. (1973) · IBM Journal of Research and Development · doi:10.1147/rd.176.0525 · Cited by 3,679
Simple Heuristics That Make Us Smart
Gigerenzer, G., & Todd, P. M. (1999) · Cited by 3,550
(2000) · doi:10.1017/cbo9780511803475 · Cited by 2,987
Irreversibility and heat generation in the computing process
Landauer, R. (2000) · IBM Journal of Research and Development · doi:10.1147/rd.441.0261 · Cited by 2,585
Quantal Response Equilibria for Normal Form Games
McKelvey, R. D., & Palfrey, T. R. (1995) · Games and Economic Behavior · doi:10.1006/game.1995.1023 · Cited by 2,545
Pfanzagl, J., & Luce, R. D. (1962) · Econometrica · doi:10.2307/1911299 · Cited by 2,469
Behavioral Game Theory. Experiments in Strategic Interaction
Tyran, J.-R. (2003) · The Journal of Socio-Economics · doi:10.1016/j.socec.2003.10.009 · Cited by 2,222
Thermodynamics and an Introduction to Thermostatistics
Callen, H. B., & Griffiths, R. B. (1987) · American Journal of Physics · doi:10.1119/1.14986 · Cited by 2,101
The thermodynamics of computation—a review
Bennett, C. H. (1982) · International Journal of Theoretical Physics · doi:10.1007/bf02084158 · Cited by 2,006
The analysis of visual motion: a comparison of neuronal and psychophysical performance
Britten, K., Shadlen, M., Newsome, W., & Movshon, J. (1992) · The Journal of Neuroscience · doi:10.1523/jneurosci.12-12-04745.1992 · Cited by 1,958
Bogacz, R., Brown, E., Moehlis, J., Holmes, P., & Cohen, J. D. (2006) · Psychological Review · doi:10.1037/0033-295x.113.4.700 · Cited by 1,866
The free-energy principle: a rough guide to the brain?
Friston, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.04.005 · Cited by 1,831
Optimality principles in sensorimotor control
Todorov, E. (2004) · Nature Neuroscience · doi:10.1038/nn1309 · Cited by 1,746
Glover, K., & Doyle, J. C. (1988) · Systems & Control Letters · doi:10.1016/0167-6911(88)90055-2 · Cited by 1,411
Bounded Rationality: The Adaptive ToolboxPreprint
Gigerenzer, G., & Selten, R. (2001) · RePEc: Research Papers in Economics · Cited by 1,367
Pattern Recognition and Machine Learning
Maindonald, J. (2007) · Journal of Statistical Software · doi:10.18637/jss.v017.b05 · Cited by 1,349
Blink: the power of thinking without thinking
(2005) · Choice Reviews Online · doi:10.5860/choice.42-6795 · Cited by 1,313
The decision and the organization
MacDougall, C. D., & Wilson, H. (1965) · Marzani & Munsell eBooks · Cited by 1,169
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · cited 1×
“) required for searching the optimum [3, 4].”
Ambiguity Aversion, Robustness, and the Variational Representation of Preferences
Maccheroni, F., Marinacci, M., & Rustichini, A. (2006) · Econometrica · doi:10.1111/j.1468-0262.2006.00716.x · Cited by 1,124
Bounded Rationality: The Adaptive Toolbox
Sent, E.-M. (2002) · Journal of Economic Issues · doi:10.1080/00213624.2002.11506464 · Cited by 1,059
H/sup ∞/-0ptimal Control and Related Minimax Design Problems: A Dynamic Game Approach
Basar, T., & Bernhard, P. (1996) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1996.536519 · Cited by 1,047
Rubinstein, A. (1998) · The MIT Press eBooks · doi:10.7551/mitpress/4702.001.0001 · Cited by 1,019
Jacobson, D. H. (1973) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1973.1100265 · Cited by 708
Friedman, D., & Rubinstein, A. (1998) · Southern Economic Journal · doi:10.2307/1060679 · Cited by 658
Quantal Response Equilibria for Extensive Form Games
McKelvey, R. D., & Palfrey, T. R. (1998) · Experimental Economics · doi:10.1023/a:1009905800005 · Cited by 656
Risk-sensitive optimal control
Whittle, P. J. L. (1990) · Medical Entomology and Zoology · Cited by 641
Hong, C. S. (1983) · Econometrica · doi:10.2307/1912052 · Cited by 629
Tribus, M., & McIrvine, E. C. (1971) · Scientific American · doi:10.1038/scientificamerican0971-179 · Cited by 607
The Foundations of Expected Utility
Fishburn, P. C. (1982) · Springer eBooks · doi:10.1007/978-94-017-3329-8 · Cited by 516
Fudenberg, D., & Kreps, D. M. (1993) · Games and Economic Behavior · doi:10.1006/game.1993.1021 · Cited by 485
Relative Entropy Policy Search
Peters, J., Mulling, K., & Altun, Y. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7727 · Cited by 455
Efficient computation of optimal actions
Todorov, E. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0710743106 · Cited by 417
Utility of Gains and Losses: Measurement-Theoretical and Experimental Approaches
Luce, R. D. (2000) · Medical Entomology and Zoology · Cited by 396
Maximum-Entropy and Bayesian Methods in Inverse Problems
(1985) · Springer eBooks · doi:10.1007/978-94-017-2221-6 · Cited by 338
Motor control is decision-making
Wolpert, D. M., & Landy, M. S. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.05.003 · Cited by 336
Linearly-solvable Markov decision problems
Todorov, E. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0176 · Cited by 334
Risk-Sensitivity in Sensorimotor Control
Braun, D. A., Nagengast, A. J., & Wolpert, D. M. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00001 · Cited by 325
Self-citationBounded Rationality and Industrial Organization
Spiegler, R. (2011) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195398717.001.0001 · Cited by 316
Advances in Prospect Theory: Cumulative Representation of Uncertainty
Tversky, A., & Kahneman, D. (2000) · Choices, Values, and Frames · doi:10.1017/cbo9780511803475.004 · Cited by 307
Linear Theory for Control of Nonlinear Stochastic Systems
Kappen, H. J. (2005) · Physical Review Letters · doi:10.1103/physrevlett.95.200201 · Cited by 299
Optimal control as a graphical model inference problem
Kappen, H. J., Gómez, V., & Opper, M. (2012) · Machine Learning · doi:10.1007/s10994-012-5278-7 · Cited by 297
The Feynman Lectures on Computation
Hey, T., & Allen, R. (1996) · ePrints Soton (University of Southampton) · Cited by 289
Axiomatic Characterizations of Information Measures
Csiszár, I. (2008) · Entropy · doi:10.3390/e10030261 · Cited by 282
A Generalized Path Integral Control Approach to Reinforcement Learning
TheodorouEvangelos, BuchliJonas, & SchaalStefan (2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953033 · Cited by 280
Information Theory of Decisions and Actions
Tishby, N., & Polani, D. (2011) · Perception-Action Cycle · doi:10.1007/978-1-4419-1452-1_19 · Cited by 267
Provably Bounded-Optimal Agents
Russell, S. J., & Subramanian, D. (1995) · Journal of Artificial Intelligence Research · doi:10.1613/jair.133 · Cited by 252
Survey of decision field theory
Busemeyer, J. R., & Diederich, A. (2002) · Mathematical Social Sciences · doi:10.1016/s0165-4896(02)00016-1 · Cited by 245
Statistical decision theory and trade-offs in the control of motor response
Maloney, L., Landy, M., & Trommershäuser, J. (2003) · Spatial Vision · doi:10.1163/156856803322467527 · Cited by 232
The probabilistic nature of preferential choice.
Rieskamp, J. (2008) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0013646 · Cited by 225
Rationality and Bounded Rationality
Aumann, R. J. (1997) · Games and Economic Behavior · doi:10.1006/game.1997.0585 · Cited by 208
The Mathematical Theory of Communication
Hockett, C. F., Shannon, C. L., & Weaver, W. (1953) · Language · doi:10.2307/410457 · Cited by 200
Rationality and Bounded Rationality
Aumann, R. J. (1997) · Cooperation: Game-Theoretic Approaches · doi:10.1007/978-3-642-60454-6_15 · Cited by 199
Information Theory ― The Bridge Connecting Bounded Rational Game Theory and Statistical Physics
Wolpert, D. H. (2007) · Springer eBooks · doi:10.1007/3-540-32834-3_12 · Cited by 150
Economic decision-making compared with an equivalent motor task
Wu, S., Delgado, M. R., & Maloney, L. T. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0900102106 · Cited by 138
Dynamics of Bayesian updating with dependent data and misspecified models
Shalizi, C. R. (2009) · Electronic Journal of Statistics · doi:10.1214/09-ejs485 · Cited by 121
A Minimum Relative Entropy Principle for Learning and Acting
Ortega, P. A., & Braun, D. A. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3062 · Cited by 117
Self-citationEntropy formulation of optimal and adaptive control
Saridis, G. N. (1988) · IEEE Transactions on Automatic Control · doi:10.1109/9.1287 · Cited by 110
Model‐based learning and the contribution of the orbitofrontal cortex to the model‐free world
McDannald, M. A., Takahashi, Y. K., Lopatina, N., Pietras, B. W., Jones, J. L., & Schoenbaum, G. M. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07982.x · Cited by 100
Dynamics of Bayesian updating with dependent data and misspecified models
Shalizi, C. R. (2009) · doi:10.1184/r1/6586577.v1 · Cited by 100
Risk-Sensitive Optimal Feedback Control Accounts for Sensorimotor Behavior under Uncertainty
Nagengast, A. J., Braun, D. A., & Wolpert, D. M. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000857 · Cited by 88
Self-citationInformation, Utility and Bounded Rationality
Ortega, D. A., & Braun, P. A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-22887-2_28 · Cited by 73
Information-theoretic approach to interactive learning
Still, S. (2009) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/85/28005 · Cited by 72
Risk-sensitivity and the mean-variance trade-off: decision making in sensorimotor control
Nagengast, A. J., Braun, D. A., & Wolpert, D. M. (2011) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2010.2518 · Cited by 59
Self-citationPath integral control and bounded rationality
Braun, D. A., Ortega, P. A., Theodorou, E. A., & Stefan Schaal, A. (2011) · IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2011.5967366 · Cited by 58
Self-citation(2010) · Uncertainty in Artificial Intelligence · Cited by 56
Time, space, and energy in reversible computing
Vitanyi, P. M. B. (2005) · ACM International Conference on Computing Frontiers (CF) · doi:10.1145/1062261.1062335 · Cited by 54
Risk sensitivity in a motor task with speed-accuracy trade-off
Nagengast, A. J., Braun, D. A., & Wolpert, D. M. (2011) · Journal of Neurophysiology · doi:10.1152/jn.00804.2010 · Cited by 48
Self-citationHysteresis effects of changing the parameters of noncooperative games
Wolpert, D. H., Harré, M., Olbrich, E., Bertschinger, N., & Jost, J. (2012) · Physical Review E · doi:10.1103/physreve.85.036102 · Cited by 48
Risk Sensitive Path Integral ControlPreprint
van den Broek, B., Wiegerinck, W., & Kappen, H. J. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1203.3523 · Cited by 47
Russell, S. (2002) · Oxford University Press eBooks · doi:10.1093/0195147669.003.0002 · Cited by 45
Jaynes, E. T. (1985) · Springer eBooks · doi:10.1007/978-94-017-2221-6_22 · Cited by 35
Strategic Choice of Preferences: the Persona Model
Wolpert, D. H., Jamison, J., Newth, D., & Harré, M. (2011) · The B E Journal of Theoretical Economics · doi:10.2202/1935-1704.1593 · Cited by 30
Model-Based Reinforcement Learning as Cognitive Search: Neurocomputational Theories
Todd, P. M., Hills, T. T., & Robbins, T. W. (2012) · Cited by 24
Russell, S. (1999) · Applied logic series · doi:10.1007/978-94-015-9204-8_2 · Cited by 21
A conversion between utility and information
Ortega, P. A., & Braun, D. A. (2010) · doi:10.2991/agi.2010.10 · Cited by 20
Self-citationRisk-Sensitivity in Bayesian Sensorimotor Integration
Grau-Moya, J., Ortega, P. A., & Braun, D. A. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002698 · Cited by 19
Self-citationA unified framework for resource-bounded autonomous agents interacting with unknown environments
Ortega, P. A. (2011) · Apollo (University of Cambridge) · doi:10.17863/cam.14005 · Cited by 13
Self-citation
Cited by
Active inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156
Toward a rational and mechanistic account of mental effort
Shenhav, A., Musslick, S., Lieder, F., Kool, W., Griffiths, T. L., Cohen, J. D., & Botvinick, M. M. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031526 · Cited by 1,077
Active inference and epistemic value
Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., & Pezzulo, G. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1020053 · Cited by 731
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., ODoherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674
A free energy principle for biological systems
Karl, F. (2012) · Entropy · doi:10.3390/e14112100 · Cited by 377
Metadata
- Authors
- Ortega, P. A., & Braun, D. A.
- Venue
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences
- Publisher
- The Royal Society
- Published
- 2013-05-08
- Volume
- 469
- Issue
- 2153
- Pages
- 20120683
- ISSN
- 1364-5021 (ISSN-L)
- 1471-2946
- OpenAlex
- W2101692583
- Cited by
- 255
- References
- 106
- FWCI
- 12.05
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.