The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.
Soft MPCritic: Value-Based Reinforcement Learning for Non-Differentiable Predictive Control
Abstract
Reinforcement learning (RL) and model predictive control (MPC) offer complementary strengths, yet existing MPC-RL frameworks commonly rely on gradient-based optimization and therefore cannot handle the non-differentiable dynamics and objectives common in RL. We propose Soft MPCritic, an MPC-RL framework that learns a soft value function and uses sampling-based planning both for online control and for generating value targets. Specifically, Soft MPCritic instantiates MPC with model predictive path integral control (MPPI) and learns a terminal Q-function through fitted value iteration. Because the value targets are computed by the same planner used for control, the learned value function is aligned with the planner and implicitly extends its effective horizon. To keep target generation tractable, we introduce an amortized warm-start strategy that reuses the open-loop action sequences planned during online control to initialize batched MPPI computations of value targets, reducing computational cost without sacrificing solution quality. For robustness to model error, Soft MPCritic plans over scenarios drawn from an ensemble of dynamics models trained for next-step prediction. On both classical and constrained control tasks, Soft MPCritic learns effective policies with non-differentiable objectives, particularly where finite-horizon MPPI alone falls short. Soft MPCritic thus offers a practical blueprint for synthesizing MPC policies directly from black-box objectives, without differentiable surrogates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1805.00909 · Cited by 366
Benchmarking Model-Based Reinforcement Learning
Wang, T., Bao, X., Clavera, I., Hoang, J., Wen, Y., Langlois, E., Zhang, S., Zhang, G., Abbeel, P., & Ba, J. (2019) · arXiv · doi:10.48550/arxiv.1907.02057 · Cited by 226
Cited by
Still being checked.
Metadata
- Authors
- Banker, T., Lawrence, N. P., & Mesbah, A.
- Venue
- arXiv
- Published
- 2026
- arXiv
- 2604.01477
- References
- 2
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Soft MPCritic: Value-Based Reinforcement Learning for Non-Differentiable Predictive Control. https://app.cochat.ai/papers/10.48550/arxiv.2604.01477
- The paper
- Banker, T., Lawrence, N. P., & Mesbah, A. (2026). Soft MPCritic: Value-Based Reinforcement Learning for Non-Differentiable Predictive Control. arXiv. https://doi.org/10.48550/arxiv.2604.01477
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.