The DOI matches its Crossref record.
Dynamic Programming
Abstract
Little has been done in the study of these intriguing questions, and I do not wish to give the impression that any extensive set of ideas exists that could be called a "theory." What is quite surprising, as far as the histories of science and philosophy are concerned, is that the major impetus for the fantastic growth of interest in brain processes, both psychological and physiological, has come from a device, a machine, the digital computer. In dealing with a human being and a human society, we enjoy the luxury of being irrational, illogical, inconsistent, and incomplete, and yet of coping. In operating a computer, we must meet the rigorous requirements for detailed instructions and absolute precision. If we understood the ability of the human mind to make effective decisions when confronted by complexity, uncertainty, and irrationality then we could use computers a million times more effectively than we do. Recognition of this fact has been a motivation for the spurt of research in the field of neurophysiology. The more we study the information processing aspects of the mind, the more perplexed and impressed we become. It will be a very long time before we understand these processes sufficiently to reproduce them. In any case, the mathematician sees hundreds and thousands of formidable new problems in dozens of blossoming areas, puzzles galore, and challenges to his heart's content. He may never resolve some of these, but he will never be bored. What more can he ask?
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The multivariate evaluation shows another benefit of deep learning models, which is the ability to handle the curse of dimensionality (Bellman, 2010; Keogh and Mueen, 2017) by leveraging different degrees of smoothness in compositional function (Poggio et al., 2017) as well as the parallel computations of the GPUs (Lu et al., 2015).”
From Deep learning for time series classification: a review (2019) · cites this paper as Bellman, 2010
In the citing paper
“if we store intermediate values of u1D439 ( u1D45B ) in memory and retrieve them whenever necessary, u1D439 ( u1D441 ) can be computed in time linear to u1D441 , achieving an exponential speedup.”
From Quantum Dynamic Programming (2025) · cites this paper as [3]
Citations
Cites
Still being checked.
Cited by
Sorted by citations
Items 101–200 of 1,021
Cyber–Physical Systems: A Perspective at the Centennial
Kim, K.-D., & Kumar, P. R. (2012) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2189792 · Cited by 771
CHOMP: Covariant Hamiltonian optimization for motion planning
Zucker, M., Ratliff, N., Dragan, A. D., Pivtoraiko, M., Klingensmith, M., Dellin, C. M., Bagnell, J. A., & Srinivasa, S. S. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913488805 · Cited by 765
Reliability optimization of series-parallel systems using a genetic algorithm
Coit, D. W., & Smith, A. E. (1996) · IEEE Transactions on Reliability · doi:10.1109/24.510811 · Cited by 759
Network Optimization: Continuous and Discrete Models
Bertsekas, D. P. (1998) · Cited by 738
Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems
Liu, D., & Wei, Q. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2281663 · Cited by 753
Timed Automata: Semantics, Algorithms and Tools
Bengtsson, J. E., & Yi, W. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-27755-2_3 · Cited by 752
Learning scheduling algorithms for data processing clusters
Mao, H., Schwarzkopf, M., Venkatakrishnan, S. B., Meng, Z., & Alizadeh, M. (2019) · ACM Special Interest Group on Data Communication · doi:10.1145/3341302.3342080 · Cited by 831
Theory and applications of adaptive control—A survey
Åström, K. J. (1983) · Automatica · doi:10.1016/0005-1098(83)90002-x · Cited by 749
An experimental design for the development of adaptive treatment strategies
Murphy, S. A. (2005) · Statistics in Medicine · doi:10.1002/sim.2022 · Cited by 746
Body Size and Consumer-Resource Dynamics
Yodzis, P., & Innes, S. L. (1992) · The American Naturalist · doi:10.1086/285380 · Cited by 731
A tutorial review of economic model predictive control methods
Ellis, M. J., Durand, H., & Christofides, P. D. (2014) · Journal of Process Control · doi:10.1016/j.jprocont.2014.03.010 · Cited by 731
Reinforcement learning in the brain
Niv, Y. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.005 · Cited by 775
A Distributional Perspective on Reinforcement LearningPreprint
Bellemare, M. G., Dabney, W., & Munos, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.06887 · Cited by 724
Martínez, C. M., Hu, X., Cao, D., Velenis, E., Gao, B., & Wellers, M. H. (2016) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2016.2582721 · Cited by 728
A review On reinforcement learning: Introduction and applications in industrial process control
Nian, R., Liu, J., & Huang, B. (2020) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2020.106886 · Cited by 755
A survey and annotated bibliography of multiobjective combinatorial optimization
Ehrgott, M., & Gandibleux, X. (2000) · OR Spectrum · doi:10.1007/s002910000046 · Cited by 701
Theory and Practice of Uncertain Programming
Liu, B. (2009) · Studies in fuzziness and soft computing · doi:10.1007/978-3-540-89484-1 · Cited by 704
Geometric Control of Mechanical Systems
(2005) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2005.860277 · Cited by 697
Best wavelet packet bases in a rate-distortion sense
Ramchandran, K., & Vetterli, M. (1993) · IEEE Transactions on Image Processing · doi:10.1109/83.217221 · Cited by 680
Murray, J. J., Cox, C. J., Lendaris, G. G., & Saeks, R. (2002) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2002.801727 · Cited by 677
Who Solved the Secretary Problem?
Ferguson, T. S. (1989) · Statistical Science · doi:10.1214/ss/1177012493 · Cited by 671
Modeling and Control of a Power-Split Hybrid Vehicle
Liu, J., & Peng, H. (2008) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2008.919447 · Cited by 667
Color image quantization for frame buffer display
Heckbert, P. S. (1982) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/800064.801294 · Cited by 666
Koenig, S., Likhachev, M. A., & Furcy, D. (2004) · Artificial Intelligence · doi:10.1016/j.artint.2003.12.001 · Cited by 662
A survey on river water quality modelling using artificial intelligence models: 2000–2020
Tiyasha, T., Tung, T. M., & Yaseen, Z. M. (2020) · Journal of Hydrology · doi:10.1016/j.jhydrol.2020.124670 · Cited by 679
Adaptive Maximum-Likelihood Receiver for Carrier-Modulated Data-Transmission Systems
Ungerboeck, G. (1974) · IRE Transactions on Communications Systems · doi:10.1109/tcom.1974.1092267 · Cited by 654
Reinforcement Learning in Healthcare: A Survey
Yu, C., Liu, J., Nemati, S., & Yin, G. (2021) · ACM Computing Surveys · doi:10.1145/3477600 · Cited by 783
Cluster analysis and mathematical programming
Hansen, P., & Jaumard, B. (1997) · Mathematical Programming · doi:10.1007/bf02614317 · Cited by 653
Zhang, H., Luo, Y., & Liu, D. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2027233 · Cited by 652
Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles
Aradi, S. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3024655 · Cited by 704
Optimal Charge Control of Plug-In Hybrid Electric Vehicles in Deregulated Electricity Markets
Rotering, N., & Ilić, M. (2010) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2010.2086083 · Cited by 640
A generic dynamic programming Matlab function
Sundstrom, O., & Guzzella, L. (2009) · The proceedings of the IEEE Conference on Control Applications/The proceedings of the ... IEEE Conference on Control Applications · doi:10.1109/cca.2009.5281131 · Cited by 636
Purely Functional Data Structures
Okasaki, C. (1998) · Cambridge University Press eBooks · doi:10.1017/cbo9780511530104 · Cited by 635
Reinforcement Learning and Markov Decision Processes
van Otterlo, M., & Wiering, M. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3_1 · Cited by 672
Job Matching and Occupational Choice
Miller, R. A. (1984) · Journal of Political Economy · doi:10.1086/261276 · Cited by 631
Semiring-based constraint satisfaction and optimization
Bistarelli, S., Montanari, U., & Rossi, F. (1997) · Journal of the ACM · doi:10.1145/256303.256306 · Cited by 631
The economics of strategic opportunity
Denrell, J. C., Fang, C., & Winter, S. G. (2003) · Strategic Management Journal · doi:10.1002/smj.341 · Cited by 613
Zhong, J.-X., Li, N., Kong, W., Liu, S., Li, T. H., & Li, G. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00133 · Cited by 626
Chapter 15 Job search and labor market analysis
Mortensen, D. T. (1986) · Handbook of labour economics · doi:10.1016/s1573-4463(86)02005-9 · Cited by 624
Dynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics
Schaal, S. (2006) · Springer eBooks · doi:10.1007/4-431-31381-8_23 · Cited by 620
Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms
Singh, S. P., Jaakkola, T., Littman, M. L., & Szepesvári, C. (2000) · Machine Learning · doi:10.1023/a:1007678930559 · Cited by 637
The image foresting transform: theory, algorithms, and applications
Falcão, A. X., Stolfi, J., & Lotufo, R. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.1261076 · Cited by 615
Psychological entropy: a framework for understanding uncertainty-related anxiety
Hirsh, J. B., Mar, R. A., & Peterson, J. B. (2012) · Psychological Review · doi:10.1037/a0026767 · Cited by 606
Statistical power for cluster analysis
Dalmaijer, E. S., Nord, C. L., & Astle, D. E. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04675-1 · Cited by 710
Rodriguez-Galiano, V. F., Chica‐Olmo, M., Abarca-Hernández, F., Atkinson, P. M., & Jeganathan, C. (2012) · Remote Sensing of Environment · doi:10.1016/j.rse.2011.12.003 · Cited by 608
ECONOMICS OF FORESTRY IN AN EVOLVING SOCIETY
Samuelson, P. A. (1976) · Economic Inquiry · doi:10.1111/j.1465-7295.1976.tb00437.x · Cited by 590
Prioritizing global conservation efforts
Wilson, K. A., McBride, M. F., Bode, M., & Possingham, H. P. (2006) · Nature · doi:10.1038/nature04366 · Cited by 589
Reinforcement Learning with Replacing Eligibility Traces
Singh, S. P., & Sutton, R. S. (1996) · Machine Learning · doi:10.1007/bf00114726 · Cited by 624
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
The Cost of Accumulating Evidence in Perceptual Decision Making
Drugowitsch, J., Moreno-Bote, R., Churchland, A. K., Shadlen, M. N., & Pouget, A. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.4010-11.2012 · Cited by 577
The economics of fishing and modern capital theory: A simplified approach
Clark, C., & Munro, G. R. (1975) · Journal of Environmental Economics and Management · doi:10.1016/0095-0696(75)90002-9 · Cited by 572
Acting Optimally in Partially Observable Stochastic Domains
Cassandra, A. R., Kaelbling, L. P., & Littman, M. L. (1994) · Cited by 570
Dynamic Price Models for New-Product Planning
Robinson, B. B., & Lakhani, C. (1975) · Management Science · doi:10.1287/mnsc.21.10.1113 · Cited by 569
Dynamic programming applications in water resources
Yakowitz, S. J. (1982) · Water Resources Research · doi:10.1029/wr018i004p00673 · Cited by 561
Optimization Concepts and Applications in Engineering
Belegundu, A. D., & Chandrupatla, T. R. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511975905 · Cited by 556
Learning to trade via direct reinforcement
Moody, J. E., & Saffell, M. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.935097 · Cited by 586
Ayvaz, S., & Alpay, K. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114598 · Cited by 616
A Survey on Generative Diffusion Models
Cao, H., Tan, C., Gao, Z., Xu, Y., Chen, G., Heng, P., & Li, S. Z. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3361474 · Cited by 834
Locally Weighted Learning for Control
Atkeson, C. G., Moore, A., & Schaal, S. (1997) · Artificial Intelligence Review · doi:10.1023/a:1006511328852 · Cited by 554
Dimension?Adaptive Tensor?Product Quadrature
Gerstner, T., & Griebel, M. (2003) · Computing · doi:10.1007/s00607-003-0015-5 · Cited by 545
Nobel Lecture: Monetary Neutrality
Lucas, R. E. (1996) · Journal of Political Economy · doi:10.1086/262037 · Cited by 542
Image completion with structure propagation
Sun, J. Z., Yuan, L., Jia, J., & Shum, H. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073274 · Cited by 541
Personality Psychology and Economics
Almlund, M., Duckworth, A. L., Heckman, J., & Kautz, T. (2011) · National Bureau of Economic Research · doi:10.3386/w16822 · Cited by 540
Why Youths Drop out of High School: The Impact of Preferences, Opportunities, and Abilities
Eckstein, Z., & Wolpin, K. I. (1999) · Econometrica · doi:10.1111/1468-0262.00081 · Cited by 540
Application of Interior-Point Methods to Model Predictive Control
Rao, C. V., Wright, S. J., & Rawlings, J. B. (1998) · Journal of Optimization Theory and Applications · doi:10.1023/a:1021711402723 · Cited by 537
Model-based Reinforcement Learning: A Survey
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2023) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000086 · Cited by 871
Net national product, wealth, and social well-being
Dasgupta, P. S., & Mäler, K.-G. (2000) · Environment and Development Economics · doi:10.1017/s1355770x00000061 · Cited by 536
Survey of Model-Based Reinforcement Learning: Applications on Robotics
Polydoros, A., & Nalpantidis, L. (2017) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-017-0468-y · Cited by 560
Control Techniques for Complex Networks
Meyn, S. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511804410 · Cited by 535
The Influence of Loyalty Programs and Short-Term Promotions on Customer Retention
Lewis, M. (2004) · Journal of Marketing Research · doi:10.1509/jmkr.41.3.281.35986 · Cited by 534
Value-Function Approximations for Partially Observable Markov Decision Processes
Hauskrecht, M. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.678 · Cited by 534
Big data for cyber physical systems in industry 4.0: a survey
Xu, L. D., & Duan, L. (2018) · Enterprise Information Systems · doi:10.1080/17517575.2018.1442934 · Cited by 564
Perseus: Randomized point-based value iteration for POMDPs
Spaan, M. T. J., & Vlassis, N. A. (2005) · UvA-DARE (University of Amsterdam) · Cited by 526
Thrun, S., & Mitchell, T. M. (1995) · Robotics and Autonomous Systems · doi:10.1016/0921-8890(95)00004-y · Cited by 536
The DRAGON system--An overview
Baker, J. (1975) · IEEE Transactions on Acoustics Speech and Signal Processing · doi:10.1109/tassp.1975.1162650 · Cited by 521
Probabilistic Algorithms and the Interactive Museum Tour-Guide Robot Minerva
Thrun, S., Beetz, M., Bennewitz, M., Burgard, W., Cremers, A. B., Dellaert, F., Fox, D., Hähnel, D., Rosenberg, C., Roy, N., et al. (2000) · The International Journal of Robotics Research · doi:10.1177/02783640022067922 · Cited by 523
Separation of estimation and control for discrete time systems
Witsenhausen, H. S. (1971) · Proceedings of the IEEE · doi:10.1109/proc.1971.8488 · Cited by 518
Aligning sentences in parallel corpora
Brown, P. F., Lai, J. C., & Mercer, R. L. (1991) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/981344.981366 · Cited by 515
The interactive museum tour-guide robot
Burgard, W., Cremers, A. B., Fox, D., Hähnel, D., Lakemeyer, G., Schulz, D., Steiner, W., & Thrun, S. (1998) · Cited by 512
On the Sample Complexity of Reinforcement Learning
Kakade, S. M. (2003) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 510
Online Planning Algorithms for POMDPs
Ross, S., Pineau, J., Paquet, S., & Chaib-draa, B. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2567 · Cited by 503
Model learning for robot control: A survey
Nguyen-Tuong, D., & Peters, J. (2011) · Cognitive Processing · doi:10.1007/s10339-011-0404-1 · Cited by 503
The Geometry of Abstraction in the Hippocampus and Prefrontal Cortex
Bernardi, S., Benna, M. K., Rigotti, M., Munuera, J., Fusi, S., & Salzman, C. D. (2020) · Cell · doi:10.1016/j.cell.2020.09.031 · Cited by 528
Probabilistic Inference and Influence Diagrams
Shachter, R. D. (1988) · Operations Research · doi:10.1287/opre.36.4.589 · Cited by 496
Recommendation for the Entropy Sources Used for Random Bit Generation
Turan, M. S., Barker, E., Kelsey, J., McKay, K. A., Baish, M. L., & Boyle, M. (2018) · NIST Special Publication 800-90B · doi:10.6028/nist.sp.800-90b · Cited by 489
Tamar, A., Wu, Y., Thomas, G., Levine, S., & Abbeel, P. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/700 · Cited by 484
Agent-Oriented Supply-Chain Management
Fox, M. S., Barbuceanu, M., & Teigen, R. (2001) · Springer eBooks · doi:10.1007/978-1-4615-1599-9_5 · Cited by 482
The neural substrates of reward processing in humans: the modern role of FMRI
McClure, S. M., York, M. K., & Montague, P. R. (2004) · The Neuroscientist · doi:10.1177/1073858404263526 · Cited by 480
Fomin, F. V., & Kratsch, D. (2010) · Texts in theoretical computer science · doi:10.1007/978-3-642-16533-7 · Cited by 507
Current methods of gene prediction, their strengths and weaknesses
Mathé, C. (2002) · Nucleic Acids Research · doi:10.1093/nar/gkf543 · Cited by 478
Roberto Bagnara, A., Hill, P. M., & Enea Zaffanella, A. (2008) · Science of Computer Programming · doi:10.1016/j.scico.2007.08.001 · Cited by 477
Tagging English text with a probabilistic model
Mérialdo, B. (1994) · Cited by 474
Theories of Decision-Making in Economics and Behavioural Science
Simon, H. A. (1966) · Palgrave Macmillan UK eBooks · doi:10.1007/978-1-349-00210-8_1 · Cited by 473
Model-based clustering of high-dimensional data: A review
Bouveyron, C., & Brunet-Saumard, C. (2012) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2012.12.008 · Cited by 470
Path Planning for Autonomous Mobile Robots: A Review
Sánchez-Ibáñez, J. R., Pérez-del-Pulgar, C. J., & Garcia-Cerezo, A. J. (2021) · Sensors · doi:10.3390/s21237898 · Cited by 485
Distributional Reinforcement Learning With Quantile Regression
Dabney, W., Rowland, M., Bellemare, M., & Munos, R. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11791 · Cited by 612
Rodriguez-Galiano, V. F., Mendes, M. P., Garcia‐Soldado, M. J., Chica‐Olmo, M., & Ribeiro, L. (2014) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2014.01.001 · Cited by 479
Zhang, H., Cui, L., & Luo, Y. (2012) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2203336 · Cited by 465
Simulation–Optimization Modeling: A Survey and Potential Application in Reservoir Systems Operation
Rani, D., & Moreira, M. M. (2009) · Water Resources Management · doi:10.1007/s11269-009-9488-0 · Cited by 476
OSPF: Anatomy of an Internet Routing Protocol
Moy, J. (1998) · CERN Document Server (European Organization for Nuclear Research) · Cited by 461
Metadata
- Authors
- Bellman, R. E.
- Affiliations
- University of Southern California (US)
- Corresponding author
- Bellman, Richard E.
- Venue
- Science
- Publisher
- Princeton University Press
- Published
- 2010-12-31
- Volume
- 153
- Issue
- 3731
- Pages
- 34-37
- ISSN
- 0036-8075 (ISSN-L)
- OpenAlex
- W2341171179
- Cited by
- 14,588
- References
- 0
- Citations by year
- 2026: 227 · 2025: 367 · 2024: 358 · 2023: 345 · 2022: 332 · 2021: 629 · 2020: 582 · 2019: 575 · 2018: 469 · 2017: 429 · 2016: 440 · 2015: 570 · 2014: 616 · 2013: 615 · 2012: 634
- FWCI
- 137.95
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- Princeton University Press eBooks (publishedVersion)
- PubMed (publishedVersion)
- Virtual Defense Library (Ministerio de Defensa) (submittedVersion)
- CERN Document Server (European Organization for Nuclear Research) (submittedVersion)
- International Linear Collider (submittedVersion)
- Andalas University Repository (Andalas University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Cognitive Computing and Networks
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Complexity and Algorithms in Graphs
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- dynamic programming, optimization, decision-making, human cognition, information processing, brain processes, neurophysiology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
