The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Model-Based Policy Search Using Monte Carlo Gradient Estimation With Real Systems Application
Open accessAbstract
In this article, we present a model-based reinforcement learning (MBRL) algorithm namedMonte Carlo probabilistic inference for learning control(MC-PILCO). This algorithm relies on Gaussian processes (GPs) to model the system dynamics and on a Monte Carlo approach to estimate the policy gradient. This defines a framework in which we ablate the choice of the components, which are the selection of the cost function, the optimization of policies using dropout, and an improved data efficiency through the use of structured kernels in the GP models. The combination of the aforementioned aspects affects dramatically the performance of MC-PILCO. Numerical comparisons in a simulated cart–pole environment show that MC-PILCO exhibits better data efficiency and control performance w.r.t. state-of-the-art GP-based MBRL algorithms. Finally, we apply MC-PILCO to real systems, considering, in particular, systems with partially measurable states. We discuss the importance of modeling both the measurement system and the state estimators during policy optimization. The effectiveness of the proposed solutions has been tested in simulation and on two real systems, which are the Furuta pendulum and the ball-and-plate rig.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Alternatively, a probabilistic model is able to eliminate most of the bias associated with predictions.”
From Improving Controller Generalization with Dimensionless Markov Decision Processes (2025) · cites this paper as Amadio et al. 2022
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
Research works printed but not linked: 40
- neuhäuser, 2011, Wilcoxon-Mann-Whitney test
- paszke, 0, Automatic differentiation in PyTorch, Proc Neural Inf Process Syst Workshop Autodiff
- 10.1162/089976602317250933 · DOI
- candela, 2005, A unifying view of sparse approximate Gaussian process regression, J Mach Learn Res, 6, 1935
- 10.1109/TPAMI.2013.218 · DOI
- 10.1016/j.ifacol.2020.12.179 · DOI
- 10.1109/TSP.2005.863042 · DOI
- 10.1017/S0962492900002804 · DOI
- 10.1109/ICRA.2019.8794229 · DOI
- 10.1038/156388a0 · DOI
- 10.1109/CDC.2016.7798708 · DOI
- 10.1109/ROBOT.2010.5509858 · DOI
- gal, 0, Improving PILCO with Bayesian neural network dynamics models, Proc Data-Efficient Mach Learn Workshop, 34
- chua, 0, Deep reinforcement learning in a handful of trials using probabilistic dynamics models, Proc Adv Neural Inf Process Syst, 4754
- 10.1109/ICRA.2015.7139550 · DOI
- 10.1109/IROS.2017.8202137 · DOI
- ng, 0, PEGASUS: A policy search method for large MDPs and POMDPs, Proc Conf Uncertainty of Artificial Intelligence, 406
- glorot, 0, Deep sparse rectifier neural networks, Proc 14th Int Conf Artif Intell Statist, 315
- rasmussen, 0, Gaussian processes in reinforcement learning, Proc Adv Neural Inf Process Syst, 751
- 10.1103/PhysRevLett.123.170602 · DOI
- williams, 2006, Gaussian Processes for Machine Learning
- deisenroth, 0, PILCO: A model-based and data-efficient approach to policy search, Proc 28th Int Conf Mach Learn, 465
- srivastava, 2014, Dropout: A simple way to prevent neural networks from overfitting, J Mach Learn Res, 15, 1929
- berkenkamp, 0, Safe model-based reinforcement learning with stability guarantees, Proc Adv Neural Inf Process Syst, 908
- 10.1109/IROS.2012.6385955 · DOI
- 10.15607/RSS.2011.VII.008 · DOI
- 10.1109/ROBOT.1997.606886 · DOI
- 10.1109/LRA.2019.2945240 · DOI
- sutton, 2018, Reinforcement Learning An Introduction
- garc?a, 2015, A comprehensive survey on safe reinforcement learning, J Mach Learn Res, 16, 1437
- parmas, 0, PIPPS: Flexible model-based policy search robust to the curse of chaos, Proc Int Conf Mach Learn, 4065
- 10.1109/LRA.2020.2977255 · DOI
- rezende, 0, Stochastic backpropagation and approximate inference in deep generative models, Proc Int Conf Mach Learn, 1278
- kingma, 0, Auto-encoding variational Bayes, Proc 2nd Int Conf Learn Representations
- 10.1155/2011/528341 · DOI
- bottou, 0, Large-scale machine learning with stochastic gradient descent, Proc COMPSTAT, 177
- 10.1109/IROS.2012.6386109 · DOI · Paper page
- 10.1109/TCST.2019.2891222 · DOI
- 10.1073/pnas.1908636117 · DOI
- 10.1115/1.3662552 · DOI · Paper page
Cited by
Improving Controller Generalization with Dimensionless Markov Decision Processes
Charvet, V., Stein, S., & Murray-Smith, R. (2025) · arXiv · doi:10.48550/arxiv.2504.10006
Metadata
- Authors
- Amadio, F., Dalla Libera, A., Antonello, R., Nikovski, D., Carli, R., & Romeres, D.
- Affiliations
- University of Padua (IT)
- Deptartment of Information Engineering, University of Padova, Padova, Italy
- Mitsubishi Electric Research Laboratories (United States) (US)
- ORCID
- Venue
- IEEE Transactions on Robotics
- Published
- 2022-12
- Volume
- 38
- Issue
- 6
- Pages
- 3879-3898
- ISSN
- 1552-3098 (ISSN-L)
- 1941-0468
- OpenAlex
- W3122949669
- Cited by
- 29
- References
- 4
- Citations by year
- 2026: 4 · 2025: 12 · 2024: 6 · 2023: 6 · 2022: 1
- FWCI
- 1.95
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Robotics (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion, other-oa)
- Padua Research Archive (University of Padova) (submittedVersion, other-oa)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Keywords
- model-based reinforcement learning, policy search, data efficiency, partially observed systems, state estimation, Furuta pendulum, ball and plate system, CartPole
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
