Q-Learning-Based Robust Control for Nonlinear Systems With Mismatched Perturbations

• Published in: IEEE transaction on neural networks and learning systems Vol. 36; no. 8; pp. 15547 - 15552
• Main Authors: Cui, Qian, Feng, Gang, Xu, Xuesong
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.08.2025
• Subjects: Artificial neural networks; Closed loop systems; Costs; Integral reinforcement learning (IRL); Learning systems; Mathematical models; mismatched perturbation; neural network (NN); Nonlinear systems; Perturbation methods; Q-function; Q-learning; Robust control; Vectors
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2025.3543336

This brief presents a novel optimal control (OC) approach based on <inline-formula> <tex-math notation="LaTeX">\mathcal {Q} </tex-math></inline-formula>-learning to address robust control challenges for uncertain nonlinear systems subject to mismatched perturbations. Unlike conventional methodologies that solve the robust control problem directly, our approach reformulates the problem by minimizing a value function that integrates perturbation information. The <inline-formula> <tex-math notation="LaTeX">\mathcal {Q} </tex-math></inline-formula>-function is subsequently constructed by coupling the optimal value function with the Hamiltonian function. To estimate the parameters of the <inline-formula> <tex-math notation="LaTeX">\mathcal {Q} </tex-math></inline-formula>-function, an integral reinforcement learning (IRL) technique is employed to develop a critic neural network (NN). Leveraging this parameterized <inline-formula> <tex-math notation="LaTeX">\mathcal {Q} </tex-math></inline-formula>-function, we derive a model-free OC solution that generalizes the model-based formulation. Furthermore, using Lyapunov's direct method, the resulting closed-loop system is guaranteed to have uniform ultimate bounded stability. A case study is presented to showcase the effectiveness and applicability of the proposed approach.