CoChat

Path integrals and symmetry breaking for optimal control theory

Open access
Kappen, H J · 2005 · Journal of Statistical Mechanics Theory and Experiment

The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.

Abstract

This paper considers linear-quadratic control of a non-linear dynamical system subject to arbitrary cost. I show that for this class of stochastic control problems the non-linear Hamilton–Jacobi–Bellman equation can be transformed into a linear equation. The transformation is similar to the transformation used to relate the classical Hamilton–Jacobi equation to the Schrödinger equation. As a result of the linearity, the usual backward computation can be replaced by a forward diffusion process that can be computed by stochastic integration or by the evaluation of a path integral. It is shown how in the deterministic limit the Pontryagin minimum principle formalism is recovered. The significance of the path integral approach is that it forms the basis for a number of efficient computational methods, such as Monte Carlo sampling, the Laplace approximation and the variational approximation. We show the effectiveness of the first two methods in a number of examples. Examples are given that show the qualitative difference between stochastic and deterministic control and the occurrence of symmetry breaking as a function of the noise.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1088/1742-5468/2005/11/p11011

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 22 references

Cited by

  • Deep Reinforcement Learning: A Brief Survey

    Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733

    “The idea of constraining policy gradient updates was explored earlier through the use of “natural gradient” updates [46] and also by means of the Kullback-Leibler (KL) divergence [48, 88].”
  • A Brief Survey of Deep Reinforcement Learning

    Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733

    “The idea of constraining policy gradient updates was explored earlier through the use of “natural gradient” updates [46] and also by means of the Kullback-Leibler (KL) divergence [48, 88].”
  • Reinforcement learning in robotics: A survey

    Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160

    “In recent years, general reinforcement learning has yielded three kinds of policy search approaches that have translated particularly well into the domain of robotics: (i) policy gradients approaches based on likelihood-ratio estimation (Sutton et al, 2000), (ii) policy updates inspired by expectation maximization (To…”
    Background
  • Soft Actor-Critic Algorithms and Applications

    Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., et al. (2018) · arXiv · doi:10.48550/arxiv.1812.05905 · Cited by 1,994

    “Optimization problems of this type have been explored in a number of prior works (Kappen, 2005; Todorov, 2007; Ziebart et al., 2008).”
    Background
  • Robotics: Science and Systems VIII

    (2013) · The MIT Press eBooks · doi:10.7551/mitpress/9816.001.0001 · Cited by 1,028

See all 481

Metadata

Authors
Kappen, H. J.
Affiliations
Radboud University Nijmegen (NL)
Corresponding author
Kappen, H J
Venue
Journal of Statistical Mechanics Theory and Experiment
Published
2005-11-01
Volume
2005
Issue
11
Pages
P11011-P11011
ISSN
1742-5468 (ISSN-L)
OpenAlex
W2026659355
Cited by
309
References
22
Citations by year
2026: 12 · 2025: 24 · 2024: 14 · 2023: 23 · 2022: 19 · 2021: 27 · 2020: 31 · 2019: 16 · 2018: 19 · 2017: 12 · 2016: 12 · 2015: 17 · 2014: 21 · 2013: 16 · 2012: 16
FWCI
2.22
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Statistical Mechanics Theory and Experiment (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • CERN Document Server (European Organization for Nuclear Research) (submittedVersion)
  • Radboud Repository (Radboud University) (submittedVersion)
  • Radboud Repository (Radboud University) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
  • Social Sciences › Economics, Econometrics and Finance › Finance › Stochastic processes and financial applications
  • Physical Sciences › Mathematics › Numerical Analysis › Numerical Solvers for Ordinary Differential Equations
Keywords
path integral, optimal control, linear quadratic control, stochastic control, Hamilton-Jacobi-Bellman equation, symmetry breaking, Pontryagin maximum principle, forward diffusion, Monte Carlo sampling, Laplace approximation, variational approximation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Path integrals and symmetry breaking for optimal control theory. https://app.cochat.ai/papers/10.1088/1742-5468/2005/11/p11011
The paper
Kappen, H. J. (2005). Path integrals and symmetry breaking for optimal control theory. Journal of Statistical Mechanics Theory and Experiment. https://doi.org/10.1088/1742-5468/2005/11/p11011

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.