On the theory of dynamic programming
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The target G-value equation is an application of Bellman's equation to the expected free energy.
“Note, the above Equation is an application of Bellman's equation (Bellman, 1952) to the expected free energy.”
From Deconstructing Deep Active Inference: A Contrarian Information Gatherer (2024) · cites this paper as Bellman, 1952
Storing intermediate values in memory allows the N-th Fibonacci number to be computed in time linear to N, achieving an exponential speedup over direct recursive computation.
“if we store intermediate values of u1D439 ( u1D45B ) in memory and retrieve them whenever necessary, u1D439 ( u1D441 ) can be computed in time linear to u1D441 , achieving an exponential speedup.”
From Quantum Dynamic Programming (2025) · cites this paper as [2]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 1
- Wald, A., Statistical Decision Functions, John Wiley & Sons, NewYork, 1950.
Cited by
Sorted by citations
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · see citing works
Machine Learning for Fluid Mechanics
Brunton, S. L., Noack, B. R., & Koumoutsakos, P. (2020) · Annual Review of Fluid Mechanics · doi:10.1146/annurev-fluid-010719-060214 · Cited by 3,063 · see citing works
Action and behavior: a free-energy formulation
Friston, K. J., Daunizeau, J., Kilner, J., & Kiebel, S. J. (2010) · Biological Cybernetics · doi:10.1007/s00422-010-0364-z · Cited by 866 · see citing works
“In control theory, value acts as a guiding function by establishing gradients, which the agent can ascend (Bellman 1952; Sutton and Barto 1981; Friston et al. 1994).”
BackgroundFriston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., O⿿Doherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674 · see citing works
Psychological entropy: a framework for understanding uncertainty-related anxiety
Hirsh, J. B., Mar, R. A., & Peterson, J. B. (2012) · Psychological Review · doi:10.1037/a0026767 · Cited by 606 · see citing works
Reinforcement learning or active inference?
Friston, K. J., Daunizeau, J., & Kiebel, S. J. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006421 · Cited by 443 · see citing works
Dopamine, affordance and active inference
Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., Dolan, R. J., Moran, R., Stephan, K. E., & Bestmann, S. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002327 · Cited by 381 · see citing works
What is optimal about motor control?
Friston, K. (2011) · Neuron · doi:10.1016/j.neuron.2011.10.018 · Cited by 332 · see citing works
Mapping value based planning and extensively trained choice in the human brain
Wunderlich, K., Dayan, P., & Dolan, R. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3068 · Cited by 310 · see citing works
Emotion and decision-making: affect-driven belief systems in anxiety and depression
Paulus, M. P., & Yu, A. J. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.07.009 · Cited by 287 · see citing works
Automated measurement of fetal head circumference using 2D ultrasound images
van den Heuvel, T. L. A., de Bruijn, D., de Korte, C. L., & Ginneken, B. V. (2018) · PLOS ONE · doi:10.1371/journal.pone.0200412 · Cited by 280 · see citing works
Active inference and agency: optimal control without cost functions
Friston, K., Samothrakis, S., & Montague, R. (2012) · Biological Cybernetics · doi:10.1007/s00422-012-0512-8 · Cited by 241 · see citing works
Planning and navigation as active inference
Kaplan, R., & Friston, K. J. (2018) · Biological Cybernetics · doi:10.1007/s00422-018-0753-2 · Cited by 203 · see citing works
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021) · IEEE Access · doi:10.1109/access.2021.3126658 · Cited by 200 · see citing works
“Reinforcement Learning (RL) can be employed to perform optimal data-driven control without the need to rely on a mathematical model of system dynamics [1], [23], [24].”
MethodScene Construction, Visual Foraging, and Active Inference
Mirza, M. B., Adams, R. A., Mathys, C. D., & Friston, K. J. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00056 · Cited by 165 · see citing works
Computational and dynamic models in neuroimaging
Friston, K. J., & Dolan, R. J. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.068 · Cited by 131 · see citing works
Hierarchical Reinforcement Learning: A Survey and Open Research Challenges
Hutsebaut-Buysse, M., Mets, K., & Latré, S. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4010009 · Cited by 129 · see citing works
Towards Revealing the Mystery behind Chain of Thought: a Theoretical Perspective
Feng, G., Zhang, B., Gu, Y., Ye, H., He, D., & Wang, L. (2023) · arXiv · doi:10.48550/arxiv.2305.15408 · Cited by 69 · see citing works
“Focusing on two basic mathematical problems as well as Dynamic Programming, we show that a bounded-depth Transformer without CoT struggles with these tasks unless its size grows prohibitively large.”
BackgroundQuantum adiabatic algorithm design using reinforcement learning
Lin, J., Lai, Z. Y., & Li, X. (2020) · Physical Review A · doi:10.1103/physreva.101.052327 · Cited by 39 · see citing works
Optimal control of gene regulatory networks for morphogen-driven tissue patterning
Pezzotta, A., & Briscoe, J. (2023) · Cell Systems · doi:10.1016/j.cels.2023.10.004 · Cited by 34 · see citing works
Son, J., Gluza, M., Takagi, R., & Ng, N. H. Y. (2025) · Physical Review Letters · doi:10.1103/physrevlett.134.180602 · Cited by 6 · see citing works
Deconstructing Deep Active Inference: A Contrarian Information Gatherer
Champion, T., Grześ, M., Bonheme, L., & Bowman, H. (2024) · Neural Computation · doi:10.1162/neco_a_01697 · Cited by 5 · see citing works
- Authors
- Bellman, R.
- Affiliations
- RAND Corporation (US)
- Corresponding author
- Bellman, Richard
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 1952-08
- Volume
- 38
- Issue
- 8
- Pages
- 716-719
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC1063639
- OpenAlex
- W2056653303
- Cited by
- 998
- References
- 0
- Citations by year
- 2026: 39 · 2025: 76 · 2024: 66 · 2023: 88 · 2022: 64 · 2021: 86 · 2020: 69 · 2019: 51 · 2018: 51 · 2017: 51 · 2016: 47 · 2015: 37 · 2014: 41 · 2013: 32 · 2012: 40
- FWCI
- 18.93
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Adaptive Dynamic Programming Control
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Complexity and Algorithms in Graphs
- Physical Sciences › Engineering › Control and Systems Engineering › Optimization and Mathematical Programming
- Keywords
- dynamic programming, optimal control, sequential decision making, optimization theory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
