The DOI matches its Crossref record; 133 of the 133 references it cites were found in a registry or an index.
Decision-Theoretic Planning: Structural Assumptions and Computational Leverage
Open accessAbstract
Planning under uncertainty is a central problem in the study of automated sequential decision making, and has been addressed by researchers in many different fields, including AI planning, decision analysis, operations research, control theory and economics. While the assumptions and perspectives adopted in these areas often differ in substantial ways, many planning problems of interest to researchers in these fields can be modeled as Markov decision processes (MDPs) and analyzed using the techniques of decision theory. This paper presents an overview and synthesis of MDP-related methods, showing how they provide a unifying framework for modeling many classes of planning problems studied in AI. It also describes structural properties of MDPs that, when exhibited by particular classes of problems, can be exploited in the construction of optimal or approximately optimal policies or plans. Planning problems commonly possess structure in the reward and value functions used to describe performance criteria, in the functions used to describe state transitions and observations, and in the relationships among features used to describe states, actions, rewards, and observations. Specialized representations, and algorithms employing these representations, can achieve computational leverage by exploiting these various forms of structure. Certain AI techniques -- in particular those based on the use of structured, intensional representations -- can be viewed in this way. This paper surveys several types of representations for both classical and decision-theoretic planning problems, and planning algorithms that exploit these representations in a number of different ways to ease the computational burden of constructing policies or plans. It focuses primarily on abstraction, aggregation and decomposition techniques based on AI-style representations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
(1988) · Cited by 16,582
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Graph-Based Algorithms for Boolean Function Manipulation
Bryant (1986) · IEEE Transactions on Computers · doi:10.1109/tc.1986.1676819 · Cited by 8,891
Decisions with Multiple Objectives: Preferences and Value Tradeoffs.
Tsoukias, A., Keeney, R. L., & Raiffa, H. (1994) · The Journal of the Operational Research Society · doi:10.2307/2584151 · Cited by 8,370
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316887 · Cited by 6,024
STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving
Fikes, R. E., & Nilsson, N. J. (1971) · Artificial Intelligence · doi:10.1016/0004-3702(71)90010-5 · Cited by 4,882
Floyd, R. W. (1962) · Communications of the ACM · doi:10.1145/367766.368168 · Cited by 3,998
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,515
Dynamic Programming and Markov Processes.
Freimer, M., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,378
Some Philosophical Problems from the Standpoint of Artificial Intelligence
McCarthy, J., & Hayes, P. J. (1981) · Readings in Artificial Intelligence · doi:10.1016/b978-0-934613-03-3.50033-7 · Cited by 3,279
Decomposition Principle for Linear Programs
Dantzig, G. B., & Wolfe, P. (1960) · Operations Research · doi:10.1287/opre.8.1.101 · Cited by 2,315
Introduction to linear and nonlinear programming
Polak, E. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100535 · Cited by 2,100
Introduction to linear and nonlinear programming
Luenberger, D. G. (1973) · Medical Entomology and Zoology · Cited by 1,915
The Optimal Control of Partially Observable Markov Processes over a Finite Horizon
Smallwood, R. D., & Sondik, E. J. (1973) · Operations Research · doi:10.1287/opre.21.5.1071 · Cited by 1,479
Neuro-Dynamic Programming: NDP
Bertsekas, D. P. (2001) · Encyclopedia of Optimization · doi:10.1007/0-306-48332-7_333 · Cited by 1,466
The Complexity of Markov Decision Processes
Papadimitriou, C. H., & Tsitsiklis, J. N. (1987) · Mathematics of Operations Research · doi:10.1287/moor.12.3.441 · Cited by 1,329
Shachter, R. D. (1986) · Operations Research · doi:10.1287/opre.34.6.871 · Cited by 1,250
Planning in a hierarchy of abstraction spaces
Sacerdoti, E. D. (1974) · Artificial Intelligence · doi:10.1016/0004-3702(74)90026-5 · Cited by 1,155
Introduction to dynamic systems: theory, models and applications
(1981) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(81)90069-0 · Cited by 1,122
Planning for conjunctive goals
Chapman, D. (1987) · Artificial Intelligence · doi:10.1016/0004-3702(87)90092-0 · Cited by 1,112
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00011-o · Cited by 1,104
Learning and executing generalized robot plans
Fikes, R. E., Hart, P. E., & Nilsson, N. J. (1972) · Artificial Intelligence · doi:10.1016/0004-3702(72)90051-3 · Cited by 1,019
A model for reasoning about persistence and causation
Dean, T., & Kanazawa, K. (1989) · Computational Intelligence · doi:10.1111/j.1467-8640.1989.tb00324.x · Cited by 999
Self-citationKorf, R. E. (1990) · Artificial Intelligence · doi:10.1016/0004-3702(90)90054-4 · Cited by 917
The computational complexity of propositional STRIPS planning
Bylander, T. (1994) · Artificial Intelligence · doi:10.1016/0004-3702(94)90081-7 · Cited by 865
Optimal control of Markov processes with incomplete state information
Åström, K. J. (1965) · Journal of Mathematical Analysis and Applications · doi:10.1016/0022-247x(65)90154-x · Cited by 854
A Probabilistic Approach to Concurrent Mapping and Localization for Mobile Robots
Thrun, S., Burgard, W., & Fox, D. (1998) · Machine Learning · doi:10.1023/a:1007436523611 · Cited by 789
TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play
Tesauro, G. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.215 · Cited by 787
UCPOP: a sound, complete, partial order planner for ADL
Penberthy, J. S., & Weld, D. S. (1992) · Cited by 676
(2005) · Lecture Notes in Control and Information Sciences · doi:10.1007/bfb0005213 · Cited by 674
Dynamic Programming and Markov Processes.
Kaufman, H., & Howard, R. A. (1961) · The American Mathematical Monthly · doi:10.2307/2312519 · Cited by 638
Reinforcement Learning with Hierarchies of Machines
Parr, R., & Russell, S. (1997) · Cited by 618
An Introduction to Least Commitment Planning
Weld, D. S. (1994) · doi:10.1609/aimag.v15i4.1109 · Cited by 581
Context-Specific Independence in Bayesian Networks
Boutilier, C. E., Friedman, N., Goldszmidt, M., & Koller, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.3562 · Cited by 568
Self-citationActing Optimally in Partially Observable Stochastic Domains
Cassandra, A. R., Kaelbling, L. P., & Littman, M. L. (1994) · Cited by 558
Algebraic Decision Diagrams and Their Applications
Bahar, R. I., Frohm, E. A., Gaona, C. M., Hachtel, G. D., Macii, E., Pardo, A., & Somenzi, F. (1993) · Cited by 540
Sondik, E. J. (1978) · Operations Research · doi:10.1287/opre.26.2.282 · Cited by 536
A survey of algorithmic methods for partially observed Markov decision processes
Lovejoy, W. S. (1991) · Annals of Operations Research · doi:10.1007/bf02055574 · Cited by 532
McAllester, D., & Rosenblitt, D. (1991) · Cited by 525
Sacerdoti, E. D. (1975) · Defense Technical Information Center (DTIC) · Cited by 503
A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes
Kearns, M., Mansour, Y., & Ng, A. Y. (2002) · Machine Learning · doi:10.1023/a:1017932429737 · Cited by 488
Probabilistic robot navigation in partially observable environments
Simmons, R. G., & Koenig, S. (1995) · Cited by 487
ADL: exploring the middle ground between STRIPS and the situation calculus
Pednault, E. (1989) · Principles of Knowledge Representation and Reasoning · Cited by 479
Universal plans for reactive robots in unpredictable environments
Schoppers, M. (1987) · Cited by 476
Bucket Elimination: A Unifying Framework for Probabilistic Inference
Dechter, R. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_4 · Cited by 452
Dean, T., & Wellman, M. P. (1991) · Cited by 451
Self-citationWhite, D. J. (1969) · Cited by 438
Exploiting Causal Independence in Bayesian Network Inference
Zhang, N. L., & Poole, D. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.305 · Cited by 433
On the Complexity of Solving Markov Decision Problems
Littman, M. L., Dean, T., & Kaelbling, L. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4971 · Cited by 424
Self-citationThe independent choice logic for modelling multiple agents under uncertainty
Poole, D. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00027-1 · Cited by 383
SPUDD: Stochastic Planning using Decision Diagrams
Hoey, J., St‐Aubin, R., Hu, A. J., & Boutilier, C. E. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6704 · Cited by 380
Self-citationFast Planning Through Planning Graph Analysis.
Blum, A. L., & Furst, M. L. (1995) · doi:10.21236/ada303260 · Cited by 365
Stochastic dynamic programming with factored representations
Boutilier, C., Dearden, R., & Goldszmidt, M. (2000) · Artificial Intelligence · doi:10.1016/s0004-3702(00)00033-3 · Cited by 361
Self-citationIncremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes
Cassandra, A. R., Littman, M. L., & Zhang, N. L. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.1525 · Cited by 361
Exploiting Structure in Policy Construction
Boutilier, C. E., Dearden, R., & Goldszmidt, M. (1995) · Cited by 346
Self-citationAn algorithm for probabilistic planning
Kushmerick, N., Hanks, S., & Weld, D. S. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00087-h · Cited by 346
Self-citationAlgorithms for Sequential Decision Making
Littman, M. L. (1996) · Cited by 341
A Reinforcement Learning Method for Maximizing Undiscounted Rewards
Schwartz, A. (1993) · Machine Learning Proceedings 1993 · doi:10.1016/b978-1-55860-307-3.50045-9 · Cited by 308
Computationally Feasible Bounds for Partially Observed Markov Decision Processes
Lovejoy, W. S. (1991) · Operations Research · doi:10.1287/opre.39.1.162 · Cited by 307
Macro-operators: A weak method for learning
Korf, R. E. (1985) · Artificial Intelligence · doi:10.1016/0004-3702(85)90012-8 · Cited by 287
Algebraic decision diagrams and their applications
Bahar, R. I., Frohm, E. A., Gaona, C. M., Hachtel, G. D., Macii, E., Pardo, A., & Somenzi, F. (2002) · International Conference on Computer Aided Design · doi:10.1109/iccad.1993.580054 · Cited by 281
Reinforcement Learning with Soft State Aggregation
Singh, S. P., Jaakkola, T., & Jordan, M. I. (1994) · Cited by 274
Dynamic programming and influence diagrams
Tatman, J. A., & Shachter, R. D. (1990) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/21.52548 · Cited by 266
Moore, A. W., & Atkeson, C. G. (1995) · Machine Learning · doi:10.1023/a:1022656217772 · Cited by 259
Introduction to dynamic systems: Theory, models and applications
Fairman, F. W. (1981) · Proceedings of the IEEE · doi:10.1109/proc.1981.12150 · Cited by 258
Conditional Nonlinear Planning
Peot, M. A., & Smith, D. E. (1992) · Elsevier eBooks · doi:10.1016/b978-0-08-049944-4.50027-6 · Cited by 252
Input generalization in delayed reinforcement learning: an algorithm and performance comparisons
Chapman, D., & Kaelbling, L. P. (1991) · Cited by 245
Modified Policy Iteration Algorithms for Discounted Markov Decision Problems
Puterman, M. L., & Shin, M. C. (1978) · Management Science · doi:10.1287/mnsc.24.11.1127 · Cited by 239
Hierarchical Solution of Markov Decision Processes using Macro-actions
Hauskrecht, M., Meuleau, N., Kaelbling, L. P., Dean, T., & Boutilier, C. E. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.7381 · Cited by 218
Self-citationA robust and fast action selection mechanism for planning
Bonet, B., Loerincs, G., & Geffner, H. (1997) · Cited by 216
Madani, O., Hanks, S., & Condon, A. (1999) · Cited by 213
Self-citationMarkov Decision Processes: Markov Decision Processes: Discrete Stochastic Dynamic Programming
Thomas, L. C. (1995) · Journal of the Operational Research Society · doi:10.1057/jors.1995.110 · Cited by 204
A Probabilistic Production and Inventory Problem
d'Epenoux, F. (1963) · Management Science · doi:10.1287/mnsc.10.1.98 · Cited by 201
Planning under time constraints in stochastic domains
Dean, T., Kaelbling, L. P., Kirman, J., & Nicholson, A. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(94)00086-g · Cited by 198
Self-citationProbabilistic planning with information gathering and contingent execution
Draper, D. L., Hanks, S., & Weld, D. S. (1994) · Cited by 191
Self-citationComputing optimal policies for partially observable decision processes using compact representations
Boutilier, C. E., & Poole, D. (1996) · Cited by 185
Self-citationPlanning with deadlines in stochastic domains
Dean, T., Kaelbling, L. P., Kirman, J., & Nicholson, A. E. (1993) · Cited by 183
Self-citationFinding Structure in Reinforcement Learning
Thrun, S., & Schwartz, A. (1994) · Cited by 183
Instance-Based Utile Distinctions for Reinforcement Learning with Hidden State
McCallum, R. A. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50055-4 · Cited by 181
Model minimization in Markov decision processes
Dean, T., & Givan, R. L. (1997) · Cited by 181
Self-citationA computational scheme for reasoning in dynamic probabilistic networks
Kjærulff, U. (1992) · Uncertainty in Artificial Intelligence · doi:10.1016/b978-1-4832-8287-9.50021-9 · Cited by 177
Adaptive aggregation methods for infinite horizon dynamic programming
Bertsekas, D. P., & Castanon, D. A. (1989) · IEEE Transactions on Automatic Control · doi:10.1109/9.24227 · Cited by 171
Solving very large weakly coupled Markov decision processes
Meuleau, N., Hauskrecht, M., Kim, K.-E., Peshkin, L., Kaelbling, L. P., Dean, T., & Boutilier, C. E. (1998) · Cited by 171
Self-citationApproximating optimal policies for partially observable stochastic domains
Parr, R., & Russell, S. (1995) · Cited by 145
TD Models: Modeling the World at a Mixture of Time Scales
Sutton, R. S. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50072-4 · Cited by 145
Blythe, J. (1999) · doi:10.1609/aimag.v20i2.1455 · Cited by 145
Abstraction and approximate decision-theoretic planning
Dearden, R., & Boutilier, C. E. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(96)00023-9 · Cited by 139
Self-citationDecomposition Techniques for Planning in Stochastic Domains
Dean, T., & Lin, S. (1995) · Cited by 134
Self-citationExploiting constraints in design synthesis
Finger, J. (1987) · Cited by 130
Nonmonotonic reasoning in the framework of situation calculus
Baker, A. B. (1991) · Artificial Intelligence · doi:10.1016/0004-3702(91)90004-4 · Cited by 127
Team-partitioned, opaque-transition reinforcement learning
Stone, P., & Veloso, M. (1999) · Proceedings of the third annual conference on Autonomous Agents · doi:10.1145/301136.301195 · Cited by 114
Intelligent planning: a decomposition and abstraction based approach
Yang, Q. (1997) · Cited by 114
Dean, T., Givan, R. L., & Leach, S. M. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.1533 · Cited by 109
Self-citationTheoretical results on reinforcement learning with temporally abstract options
Precup, D., Sutton, R. S., & Singh, S. P. (1998) · Lecture notes in computer science · doi:10.1007/bfb0026709 · Cited by 103
Using temporal logic to control search in a forward chaining planner
Fahiem Bacchus, A., & Kabanza, F. (1996) · Cited by 103
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., & Singh, S. P. (1996) · The MIT Press eBooks · doi:10.7551/mitpress/2026.003.0005 · Cited by 92
Using abstractions for decision-theoretic planning with time constraints
Boutilier, C. E., & Dearden, R. (1994) · Cited by 90
Self-citationA heuristic variable grid solution method for POMDPs
Brafman, R. I. (1997) · Cited by 86
Utility Models for Goal‐Directed, Decision‐Theoretic Planners
Haddawy, P., & Hanks, S. (1998) · Computational Intelligence · doi:10.1111/0824-7935.00068 · Cited by 83
Self-citationHow to Dynamically Merge Markov Decision Processes
Singh, S. P., Cohn, D. A., & Harlequin Inc (1997) · Cited by 83
Bacchus, F., Boutilier, C. E., & Grove, A. J. (1996) · Cited by 77
Self-citationRefinement Planning as a Unifying Framework for Plan Synthesis
Subbarao Kambhampati, B. (1997) · AI Magazine · doi:10.21918/aimag.v18i2.1295 · Cited by 76
Modeling a dynamic and uncertain world I
Hanks, S., & McDermott, D. (1994) · Artificial Intelligence · doi:10.1016/0004-3702(94)90002-7 · Cited by 75
Self-citationSolution Procedures for Partially Observed Markov Decision Processes
White, C. C., & Scherer, W. T. (1989) · Operations Research · doi:10.1287/opre.37.5.791 · Cited by 60
Advances in Probabilistic Reasoning
Geiger, D., & Heckerman, D. E. (1991) · Elsevier eBooks · doi:10.1016/b978-1-55860-203-8.50019-x · Cited by 55
Projecting plans for uncertain worlds
McDermott, D., & Hanks, S. (1990) · Cited by 52
Self-citationBoutilier, C. E., Brafman, R. I., & Geib, C. (1997) · Cited by 51
Self-citationTeam-Partitioned, Opaque-Transition Reinforcement Learning
Stone, P., & Veloso, M. (1999) · Lecture notes in computer science · doi:10.1007/3-540-48422-1_21 · Cited by 48
Iterative Aggregation-Disaggregation Procedures for Discounted Semi-Markov Reward Processes
Schweitzer, P. J., Puterman, M. L., & Kindle, K. W. (1985) · Operations Research · doi:10.1287/opre.33.3.589 · Cited by 48
Approximate Value Trees in Structured Dynamic Programming.
Boutilier, C. E., & Dearden, R. (1996) · International Conference on Machine Learning · Cited by 47
Self-citationControl strategies for a stochastic planner
Tash, J. K., & Russell, S. (1994) · Cited by 43
The role of relevance in explanation I: Irrelevance as statistical independence
Shimony, S. E. (1993) · International Journal of Approximate Reasoning · doi:10.1016/0888-613x(93)90027-b · Cited by 43
Structured Reachability Analysis for Markov Decision ProcessesPreprint
Boutilier, C. E., Brafman, R. I., & Geib, C. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.7361 · Cited by 39
Self-citationMahadevan, S. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50028-3 · Cited by 37
Explanation-Based Learning and Reinforcement Learning: A Unified View
Dietterich, T. G., & Flann, N. S. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50030-x · Cited by 37
Control Knowledge to Improve Plan Quality
Pérez, M. A., & Carbonell, J. (2018) · doi:10.1184/r1/6604469 · Cited by 34
Structured solution methods for non-Markovian decision processes
Bacchus, F., Boutilier, C. E., & Grove, A. J. (1997) · Cited by 34
Self-citationHeuristic search in cyclic AND/OR graphs
Hansen, E. A., & Zilberstein, S. (1998) · Cited by 34
Integrating Planning and Execution in Stochastic Domains
Dearden, R., & Boutilier, C. E. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-332-5.50026-2 · Cited by 28
Self-citationOptimal Probabilistic and Decision-Theoretic Planning using Markovian
Koenig, S. (1992) · Cited by 27
Learning Sorting and Decision Trees with POMDPs
Bonet, B., & Geffner, H. (1998) · Cited by 23
Process-oriented planning and average-reward optimality
Boutiller, C., & Puterman, M. L. (1995) · Cited by 20
The frame problem and Bayesian network action representations
Boutilier, C. E., & Goldszmidt, M. (1996) · Lecture notes in computer science · doi:10.1007/3-540-61291-2_42 · Cited by 20
Self-citationCorrelated Action Effects in Decision Theoretic RegressionPreprint
Boutilier, C. E. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.1522 · Cited by 19
Self-citationExploiting the Rule Structure for Decision Making within the Independent Choice LogicPreprint
Poole, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4978 · Cited by 16
Representing Uncertainty in Simple Planners
Goldman, R. P., & Boddy, M. (1994) · Principles of Knowledge Representation and Reasoning · doi:10.1016/b978-1-4832-1452-8.50118-4 · Cited by 13
Model minimization, regression, and propositional STRIPS planning
Givan, R. L., & Dean, T. (1997) · Cited by 13
Self-citationGenerating optimal policies for high-level plans with conditional branches and loops
Lin, S., & Dean, T. (1996) · Cited by 12
Self-citationOptimal Probabilistic and Decision-Theoretic Planning using Markovian Decision Theory
Koenig, S. (n.d.) · UC Berkeley · Cited by 8
The Frame Problem and Bayesian Network Action Representations*
Boutilier, C. E., & Goldszmidt, M. (1999) · Artificial intelligence · doi:10.1007/978-3-642-60211-5_5 · Cited by 1
Self-citation
Cited by
A Roadmap of Agent Research and Development
Jennings, N. R., Sycara, K., & Wooldridge, M. (1998) · Autonomous Agents and Multi-Agent Systems · doi:10.1023/a:1010090405266 · Cited by 2,161
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
Dietterich, T. G. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.639 · Cited by 1,435
A domain-independent framework for modeling emotion
Gratch, J., & Marsella, S. (2004) · Cognitive Systems Research · doi:10.1016/j.cogsys.2004.02.002 · Cited by 725
Reinforcement Learning and Markov Decision Processes
van Otterlo, M., & Wiering, M. (2012) · Adaptation, Learning, and Optimization · doi:10.1007/978-3-642-27645-3_1 · Cited by 628
Value-Function Approximations for Partially Observable Markov Decision Processes
Hauskrecht, M. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.678 · Cited by 527
Metadata
- Authors
- Boutilier, C., Dean, T., & Hanks, S.
- Venue
- Journal of Artificial Intelligence Research
- Publisher
- AI Access Foundation
- Published
- 1999-07-01
- Volume
- 11
- Pages
- 1-94
- ISSN
- 1076-9757 (ISSN-L)
- OpenAlex
- W2061504687
- Cited by
- 1,074
- References
- 133
- FWCI
- 54.12
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.