CoChat
Sign Up
Cited by 23Reinforcement Learning in RoboticsAI-based Problem Solving and PlanningAdvanced Bandit Algorithms Research

Planning and Learning in Environments with Delayed Feedback

Walsh, Thomas J., Nouri, Ali, Li, Lihong, Littman, Michael L.·2007·Lecture Notes in Computer Science
DOI 10.1007/978-3-540-74958-5_41
Open this paper in CoChat — free account

Integrity signals

Registry

Crossref · 10.1007/978-3-540-74958-5_41

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prior time-delayed decision-making methods in reinforcement learning are not always applicable to imitation learning, and none of them leverage action chunking.

    “Separately, in reinforcement learning, a variety of prior works have developed time-delayed decision-making methods [57, 16, 54, 63, 66, 67]. However, these approaches are not always applicable to imitation learning, and none of them leverage action chunking.”

    From Real-Time Execution of Action Chunking Flow Policies (2025) · cites this paper as [63]

Cites

No references could be resolved yet.

Research works printed but not linked: 17

  1. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA (1998)
  2. Altman, E., Nain, P.: Closed-loop control with delayed information. In: Proc. 1992 ACM SIGMETRICS and PERFORMANCE, 1-5 1992, pp. 193–204. ACM Press, New York (1992)
  3. Brooks, D.M., Leondes, C.T.: Markov decision processes with state-information lag. Operations Research 20(4), 904–907 (1972) · DOI
  4. Katsikopoulos, K.V., Engelbrecht, S.E.: Markov decision processes with delays and asynchronous cost collection. IEEE Transactions on Automatic Control 48, 568–574 (2003) · DOI · Paper page
  5. Bertsekas, D.P.: Dynamic Programming and Optimal Control, 2nd edn., vol. 1/2. Athena Scientific (2001)
  6. Bander, J.L., White III, C.C.: Markov decision processes with noise-corrupted and delayed state observations. The Journal of the Operational Research Society 50, 660–668 (1999) · DOI
  7. Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York (1994) · DOI
  8. Loch, J., Singh, S.: Using eligibility traces to find the best memoryless policy in partially observable Markov decision processes. In: ICML, pp. 323–331 (1998)
  9. Brafman, R.I., Tennenholtz, M.: R-max–a general polynomial time algorithm for near-optimal reinforcement learning. Journal of Machine Learning Research 3, 213–231 (2002) · DOI
  10. Atkeson, C.G., Moore, A.W., Schaal, S.: Locally weighted learning for control. Artificial Intelligence Review 11(1-5), 75–113 (1997) · DOI
  11. Papadimitriou, C.H., Tsitsiklis, J.N.: The complexity of Markov decision processes. Mathematics of Operations Research 12(3), 441–450 (1987) · DOI · Paper page
  12. Singh, S.P., Yee, R.C.: An upper bound on the loss from approximate optimal-value functions. Machine Learning 16(3), 227–233 (1994)
  13. Littman, M.L.: Algorithms for Sequential Decision Making. PhD thesis, Brown University, Providence, RI (1996)
  14. Munos, R., Moore, A.W.: Rates of convergence for variable resolution schemes in optimal control. In: ICML, pp. 647–654 (2000)
  15. Lin, L.J.: Reinforcement Learning for Robots using Neural Networks. PhD thesis, Carnegie Mellon University, Pittsburgh, PA (1993)
  16. Vijayakumar, S., Schaal, S.: Locally weighted projection regression: An o(n) algorithm for incremental real time learning in high dimensional space. In: ICML, pp. 1079–1086 (2000)
  17. Boyan, J.A., Moore, A.W.: Generalization in reinforcement learning: Safely approximating the value function. In: NIPS, pp. 369–376 (1995)

Cited by

Sorted by citations

Authors
Walsh, T. J., Nouri, A., Li, L., & Littman, M. L.
Affiliations
Rutgers, The State University of New Jersey (US)
ORCID
Venue
Lecture Notes in Computer Science
Publisher
Springer Berlin Heidelberg
Published
2007-09-07
Pages
442-453
ISSN
  • 0302-9743 (ISSN-L)
  • 1611-3349
DOI
10.1007/978-3-540-74958-5_41
OpenAlex
W2097951814
Cited by
23
References
0
Citations by year
2026: 1 · 2024: 3 · 2023: 2 · 2022: 1 · 2021: 1 · 2020: 1 · 2019: 1 · 2016: 1 · 2015: 5 · 2014: 2 · 2012: 1
FWCI
0.87
Type
conference paper
Language
en
Open access
closed
Locations
Lecture notes in computer science (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
Keywords
delayed observations, delayed rewards, model-based simulation, model-based reinforcement learning, continuous environments, decision-making, constrained dynamics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review