Predefined-Time Tracking Control of Servo Hydraulic Cylinder Based on Reinforcement Learning
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 2 of the 16 sources its claims cite were checked for support.
Abstract
Electro-hydraulic servo systems are characterized by significant nonlinearities. Reinforcement learning (RL), known for its model-free nature and adaptive learning capabilities, presents a promising approach for handling uncertainties inherent in such systems. This paper proposes a predefined-time tracking control scheme based on RL, which achieves fast and accurate tracking performance. The proposed design employs an actor–critic neural network strategy to actively compensate for system uncertainties. Within a conventional backstepping framework, a command-filtering technique is integrated to construct a predefined-time control structure. This not only circumvents the issue of differential explosion but also guarantees system convergence within a predefined time, which can be specified independently by the designer. Simulation results and comparisons validate the enhanced control performance of the proposed controller.
Overview
CoChat Summary
TL;DR
This paper proposes a predefined-time tracking controller for electro-hydraulic servo systems that combines actor-critic reinforcement learning with command-filtered backstepping to compensate for unknown dynamics. Simulations show improved tracking accuracy over a predefined-time controller without RL and a velocity feed-forward PI controller.
Key findings
- The proposed RLPDTC achieves a steady-state tracking error amplitude stabilizing at 0.04 mm, and the paper reports that it reduces the maximum tracking error by approximately 81% and the mean square error by about 82% relative to PDTC.
- In the last-cycle performance table, RLPDTC gives maximum error \(M_e = 3.9461 \times 10^{-5}\), mean \(\mu = 2.5966 \times 10^{-6}\), and standard deviation \(\sigma = 2.6262 \times 10^{-5}\), compared with PDTC \(M_e = 2.1107 \times 10^{-4}\), \(\mu = 2.6383 \times 10^{-6}\), and \(\sigma = 1.4840 \times 10^{-4}\).
- The actor-critic network approximates the system’s unknown dynamics, and the paper reports that it also approximates a superimposed high-frequency sinusoidal disturbance \(\sin(10t)\) while maintaining predefined-time convergence and global stability.
- The control input trajectory is reported as smooth and continuous, supporting practical applicability.
Methods
- The paper designs an RL-based predefined-time controller within a command-filtered backstepping framework for a valve-controlled double-rod hydraulic cylinder driving an inertial load, using actor and critic neural networks to compensate for unknown system dynamics and uncertainties.
- Stability analysis uses Lyapunov functions and lemmas for predefined-time practical stability; the convergence time is user-specified and independent of initial states.
- Simulations are conducted in MATLAB R2018a with system parameters including \(m = 30\) kg, \(A = 904.8\) mm\(^2\), \(B_v = 4000\) N·s/m, \(V_t = 7.96 \times 10^{-5}\) m\(^3\), \(P_s = 10\) MPa, \(\beta_e = 700\) MPa, and \(C_t = 1 \times 10^{-13}\) m\(^3\)/s/Pa; the desired trajectory is \(0.01 \cdot \cos(2t)\) m, and RLPDTC is compared with PDTC and VFPI.
Limitations
- The paper states that the controller design still relies noticeably on key system parameter information, and future work will focus on self-tuning control parameters and optimizing controller design.
- The paper notes in Remark 3 that the RL network is proven without boundedness assumptions in the mechanical channel, but extending RL to learn and compensate for uncertainties in the hydraulic dynamics is challenging because accurate differential signals of pressure dynamics are difficult to obtain reliably.
- Evident from the text: validation is simulation-only, with no experimental implementation; several RL parameters were selected through simulation trial-and-error, and the analysis assumes a twice-continuously-differentiable bounded desired trajectory and bounded system uncertainty.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 1 Not valid · 7 Not checked · 44 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“To tackle these challenges, the predefined-time control concept introduced in [29] becomes the focus of this paper, as it allows arbitrary pre-specification of the system convergence time, independent of initial states or design parameters.”
What the sources say
- Partly supportedcites [29] Loukianov, 2018, A class of predefined-time stable dynamical systems, IMA J. Math. Control Inf. sourceExperimental — accuracy not yet measured
The Introduction defines predefined-time stability by making an upper bound on settling time tunable and describes fixed-time stability as uniform across initial conditions. But the Conclusions say that bound has an explicit relationship with the system gain, a design parameter. The source therefore supports preselecting a convergence-time bound, not independence from design parameters or an identical exact convergence time for every initial state. (LLM, conf=0.88)
states a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Similarly, robust control tends to rely on increased feedback gains to suppress disturbances, which may excite unmodeled high-frequency dynamics.”
What the sources say
- Partly supportedcites [8] Yao, 2012, Robust control for static loading of electro-hydraulic load simulator with friction compensation, Chin. J. Aeronaut. sourceExperimental — accuracy not yet measured
Our copy supports only the gain part of the sentence: in the discussion following Theorem 1 (Proof of Lemma 2 section, p. ~959-960) the paper states that 'k can be made arbitrarily large, and ε/k, the bound of e₂(∞), can be made arbitrarily small by increasing gains k and/or decreasing controller parameter ε', and the robust term u_s is designed to dominate the model uncertainties Δ (Sec. 3.2, p. 959). However, no passage of our copy says that such high gains may excite unmodeled high-frequency dynamics — the paper treats large k purely as a performance advantage and never discusses unmodeled dynamics as a consequence. The only cautionary remark about large gains in our copy is about the comparison PID controller ('larger parameters will lead the system to unstable', Table 1/Section 4), which is a different controller and a different failure mode (instability, not excitation of high-frequency unmodeled dynamics). (LLM, conf=0.65)
- Not checked yetcites [9] Zhou, 2025, Performance Guaranteed Robust Tracking Control of Electro-Hydraulic Systems with Input Delays and Output Constraints, IEEE Trans. Ind. Electron. source
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Simulation results and comparisons validate the enhanced control performance of the proposed controller.”
states a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“RLPDTC ensures that the tracking error converges to a bounded range within the predefined time”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the Actor-Critic network demonstrates a considerable ability to approximate this high-frequency disturbance”
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
29 more claims
Create a free account to see all 54 claims and to verify the 14 references still open.
- It may state two separate claims as one. 9
- Probably not a substantial claim. 10
- The quote only partly states this claim. 2
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ 3.1. Preliminaries
“ζ is defined as a positive adjustable constant constrained by 0 < ζ < 1”
§ 4. Simulation Results
“r = 0.2, ζ = 1, η = 0.001, κ = 100, λ = 0.5, a 1 = 50, a 2 = 1, and Tf = 1 s.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3.3. RL-Based Predefined-Time Controller Design
“λ e 12 = ( 1 2 ) 1 + r 2 · 5 r 2 · 2 a 2 rT f a ′ .”
§ 3.3. RL-Based Predefined-Time Controller Design
“where the constant λ e 12 = 2 -r 2 ( 1 2 ) 1 -r 2 · 2 a 1 rT f a ′ .”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4. Simulation Results
“According to Table 1, RLPDTC reduces the maximum tracking error by approximately 81% and the mean square error by about 82%”
§ 4. Simulation Results
“| Indices | Me | µ | σ | |-----------|-----------------|-----------------|-----------------| | RLPDTC | 3.9461 × 10 - 5 | 2.5966 × 10 - 6 | 2.6262 × 10 - 5 | | PDTC | 2.1107 × 10 - 4 | 2.6383 × 10 - 6 | 1.4840 × 10 - 4 |”
Citations
Cites
A class of predefined-time stable dynamical systems
Sánchez-Torres, J. D., Gómez-Gutiérrez, D., López, E., & Loukianov, A. G. (2018) · IMA Journal of Mathematical Control and Information · doi:10.1093/imamci/dnx004 · Cited by 570
Improved non-singular fast terminal sliding mode control with disturbance observer for PMSM drives
Xu, B., Zhang, L., & Ji, W. (2021) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2021.3083925 · Cited by 428 · cited 1×
“Observer-based control methods, such as the Extended State Observer (ESO) [12,13] and the Disturbance Observer [14,15], circumvent high-gain feedback by estimating and proactively compensating for uncertainties.”
Reinforcement learning-based optimal tracking control of an unknown unmanned surface vehicle
Wang, N., Gao, Y., Zhao, H., & Ahn, C. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3009214 · Cited by 322 · cited 1×
“In [20], an actor–critic RL-based optimal control scheme was developed to enhance tracking performance for unmanned surface vehicles exhibiting complex unknown nonlinearities.”
A Survey on Control of Hydraulic Robotic Manipulators With Projection to Future Trends
Mattila, J., Koivumaki, J., Caldwell, D. G., & Semini, C. (2017) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2017.2668604 · Cited by 277
Cao, S., Sun, L., Jiang, J., & Zuo, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3116713 · Cited by 221 · cited 3×
“[27] combined fixed-time control with reinforcement learning for spacecraft attitude consensus control, with a similar hybrid approach applied to manipulator control in [28].”
Design and implementation of an adaptive sliding-mode dynamic controller for wheeled mobile robots
Chen, C.-Y., Li, T.-H. S., Yeh, Y.-C., & Chang, C.-C. (2009) · Mechatronics · doi:10.1016/j.mechatronics.2008.09.004 · Cited by 209 · cited 1×
“…beyond these limitations and more directly address or compensate for the system’s inherent nonlinearities, the field of advanced control has evolved toward a variety of nonlinear control strategies developed in recent decades, such as sliding mode control [7] and nonlinear robust control [8,9].”
Approximation-Free Control for Vehicle Active Suspensions With Hydraulic Actuator
Huang, Y., Na, J., Wu, X., & Gao, G. (2018) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2018.2798564 · Cited by 181
RISE-Based Asymptotic Prescribed Performance Tracking Control of Nonlinear Servo Mechanisms
Wang, S., Na, J., & Ren, X. (2018) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2017.2769683 · Cited by 159 · cited 1×
“The robust integral of the sign of the error (RISE) controller [10,11] guarantees asymptotic tracking performance owing to the strong robustness of its integral robust control law against smooth time-varying disturbances.”
Model-Based Reinforcement Learning Control of Electrohydraulic Position Servo Systems
Yao, Z., Liang, X., Jiang, G.-P., & Yao, J. (2023) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2022.3219115 · Cited by 134 · cited 1×
“Similar integrations of intelligent decision-making with system dynamic stability have been successfully demonstrated in studies [22,23].”
Self-citationNeural adaptive control of single-rod electrohydraulic system with lumped uncertainty
Guo, Q., & Chen, Z. (2021) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2020.106869 · Cited by 114
Target tracking control of UAV through deep reinforcement learning
Ma, B., Liu, Z., Zhao, W., Yuan, J., Long, H., Wang, X., & Yuan, Z. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3249900 · Cited by 91 · cited 1×
“As a representative methodology, reinforcement learning (RL) eliminates the need for explicit mathematical models by autonomously learning optimal control policies through environmental interaction [18,19].”
Robust control for static loading of electro-hydraulic load simulator with friction compensation
YAO, J., JIAO, Z., & YAO, B. (2012) · Chinese Journal of Aeronautics · doi:10.1016/s1000-9361(11)60467-6 · Cited by 89 · cited 1×
“…beyond these limitations and more directly address or compensate for the system’s inherent nonlinearities, the field of advanced control has evolved toward a variety of nonlinear control strategies developed in recent decades, such as sliding mode control [7] and nonlinear robust control [8,9].”
Self-citationESO-based robust and high-precision tracking control for aerial manipulation
Cao, H., Li, Y., Liu, C., & Zhao, S. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3260874 · Cited by 87 · cited 1×
“Observer-based control methods, such as the Extended State Observer (ESO) [12,13] and the Disturbance Observer [14,15], circumvent high-gain feedback by estimating and proactively compensating for uncertainties.”
Chen, R.-Z., Li, Y.-X., & Ahn, C. K. (2023) · Aerospace Science and Technology · doi:10.1016/j.ast.2022.108060 · Cited by 43 · cited 1×
“[27] combined fixed-time control with reinforcement learning for spacecraft attitude consensus control, with a similar hybrid approach applied to manipulator control in [28].”
Adaptive neural network finite-time tracking control for uncertain hydraulic manipulators
Liang, X., Yao, Z., Deng, W., & Yao, J. (2025) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2024.3396493 · Cited by 41 · cited 1×
“The authors of [24] developed an observer-based finite-time controller for hydraulic actuators with complex unknown dynamics, while [25] incorporated dynamic surface control to achieve high-precision finite-time control of hydraulic manipulators without complexity explosion.”
Self-citationZheng, M., Wu, Y., & Li, C. (2021) · Aerospace Science and Technology · doi:10.1016/j.ast.2021.107126 · Cited by 37 · cited 1×
“Another study [21] integrated re-inforcement learning with a sliding mode observer design, achieving spacecraft attitude stabilization control within a conventional backstepping framework.”
Kharrat, M. (2024) · Journal of Applied Mathematics and Computing · doi:10.1007/s12190-024-02042-2 · Cited by 37 · cited 1×
“Additionally, leveraging the universal function approximation property of neural networks, researchers have successfully addressed a wide range of control problems involving unknown system dynamics [16,17].”
Wang, C., Wang, J., Guo, Q., Liu, Z., & Chen, C. L. P. (2025) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2024.3429638 · Cited by 36 · cited 1×
“[26] proposed a fixed-time disturbance observer-based controller for electro-hydraulic systems under input saturation constraints.”
Disturbance observer based actor-critic learning control for uncertain nonlinear systems
Liang, X., Yao, Z., Ge, Y., & Yao, J. (2023) · Chinese Journal of Aeronautics · doi:10.1016/j.cja.2023.06.028 · Cited by 33 · cited 1×
“Similar integrations of intelligent decision-making with system dynamic stability have been successfully demonstrated in studies [22,23].”
Self-citationBai, W., Chen, D., Zhao, B., & D'Ariano, A. (2025) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3458894 · Cited by 31 · cited 1×
“As a representative methodology, reinforcement learning (RL) eliminates the need for explicit mathematical models by autonomously learning optimal control policies through environmental interaction [18,19].”
Precise position control of an electro-hydraulic servo system via robust linear approximation
Fallahi, M., Zareinejad, M., Baghestan, K., Tivay, A., Rezaei, S. M., & Abdullah, A. (2018) · ISA Transactions · doi:10.1016/j.isatra.2018.06.002 · Cited by 30 · cited 2×
“For instance, Fallahi et al. [5] modeled linearization errors in an electro-hydraulic servo system as a polytopic uncertainty and designed an H ∞ controller via Linear Matrix Inequalities, achieving robust position con-trol.”
Lu, X., Liu, W., Zhou, C., & Huang, M. (2016) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2016.2643689 · Cited by 26
Tang, G., Xue, W., Huang, X., & Song, K. (2024) · Control Engineering Practice · doi:10.1016/j.conengprac.2023.105808 · Cited by 19 · cited 1×
“Observer-based control methods, such as the Extended State Observer (ESO) [12,13] and the Disturbance Observer [14,15], circumvent high-gain feedback by estimating and proactively compensating for uncertainties.”
Accurate Finite-Time Motion Control of Hydraulic Actuators with Event-Triggered Input
Chen, J., Du, X., Lyu, L., Fei, Z., & Sun, X.-M. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3319348 · Cited by 18 · cited 1×
“The authors of [24] developed an observer-based finite-time controller for hydraulic actuators with complex unknown dynamics, while [25] incorporated dynamic surface control to achieve high-precision finite-time control of hydraulic manipulators without complexity explosion.”
Lu, K., Feng, G., & Ding, B. (2023) · Sensors · doi:10.3390/s23229092 · Cited by 14 · cited 2×
“Similarly, Lu et al. [6] addressed parameter uncertainties and load disturbances in a valve-controlled hydraulic motor by formulating them as structured norm-bounded uncertainties and developing a robust H ∞ tracking controller.”
Fixed-time adaptive neural network compensation control for uncertain nonlinear systems
Ma, J., Yao, Z., Deng, W., & Yao, J. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107563 · Cited by 11 · cited 1×
“Additionally, leveraging the universal function approximation property of neural networks, researchers have successfully addressed a wide range of control problems involving unknown system dynamics [16,17].”
Self-citationWei, X., Song, X., Zhang, H., Hu, X., & Li, X. (2026) · Applied Mathematical Modelling · doi:10.1016/j.apm.2025.116598 · Cited by 8 · cited 1×
“Observer-based control methods, such as the Extended State Observer (ESO) [12,13] and the Disturbance Observer [14,15], circumvent high-gain feedback by estimating and proactively compensating for uncertainties.”
RISE-based adaptive tracking control for Euler–Lagrange mechanical systems with matched disturbances
Liu, L., Yue, X., Wen, H., & Dai, H. (2022) · ISA Transactions · doi:10.1016/j.isatra.2022.09.045 · Cited by 7 · cited 1×
“The robust integral of the sign of the error (RISE) controller [10,11] guarantees asymptotic tracking performance owing to the strong robustness of its integral robust control law against smooth time-varying disturbances.”
Que, N., Sun, H., Deng, W., Yao, J., Zhao, X., Hu, J., & Luo, M. (2025) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.70018 · Cited by 5
Self-citationZhou, J., Hua, C., Zhang, B., Li, Y., & Zhang, Y. (2025) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2025.3579077 · Cited by 3
Research works printed but not linked: 1
- Xie, 2024, Predefined-Time Disturbance Estimation and Attitude Control for Rigid Spacecraft, IEEE Trans. Circuits Syst. II Express Briefs, 71, 2089
Cited by
Mhemdi, A., Kharrat, M., Gargouri, A., & Mercorelli, P. (2026) · Actuators · doi:10.3390/act15070384 · Cited by 2
“Fuzzy logic systems (FLSs) and neural networks (NNs), known for their ability to approximate functions universally, have been incorporated into adaptive control designs to address unknown functions, yielding many promising results [8–10].”
PI-Based Adaptive Actor–Critic Displacement Volume Control of Axial-Piston Pump
Mitov, A., Slavov, T. N., & Kralev, J. K. (2026) · Technologies · doi:10.3390/technologies14060380
Metadata
- Authors
- Han, T., Nie, X., Que, N., Lu, J., Yao, J., & Yu, X.
- Affiliations
- Aircraft Strength Research Institute (China) (CN)
- Nanjing University of Science and Technology (CN)
- Corresponding author
- Yu, Xiaochuan
- Venue
- Actuators
- Publisher
- MDPI AG
- Published
- 2025-12-24
- Volume
- 15
- Issue
- 1
- Pages
- 9
- ISSN
- 2076-0825 (ISSN-L)
- OpenAlex
- W7117327103
- Cited by
- 1
- References
- 30
- Citations by year
- 2026: 1
- FWCI
- 0.37
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Actuators (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Control of Nonlinear Systems
- Physical Sciences › Engineering › Mechanical Engineering › Hydraulic and Pneumatic Systems
- Physical Sciences › Engineering › Control and Systems Engineering › Iterative Learning Control Systems
- Keywords
- predefined-time control, electro-hydraulic servo system, reinforcement learning, actor-critic neural networks, backstepping control, command filter, explosion of complexity, system uncertainty compensation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.