Model-Free μ Synthesis via Adversarial Reinforcement Learning

• Published in: Proceedings of the American Control Conference pp. 3335 - 3341
• Main Authors: Keivan, Darioush, Havens, Aaron, Seiler, Peter, Dullerud, Geir, Hu, Bin
• Format: Conference Proceeding
• Language: English
• Published: American Automatic Control Council 08.06.2022
• Subjects: Approximation algorithms; Benchmark testing; MIMICs; Numerical models; Reinforcement learning; Robust control
• ISSN: 2378-5861
• DOI: 10.23919/ACC53348.2022.9867674

Motivated by the recent empirical success of policy-based reinforcement learning (RL), there has been a research trend studying the performance of policy-based RL methods on standard control benchmark problems. In this paper, we examine the effectiveness of policy-based RL methods on an important robust control problem, namely μ synthesis. We build a connection between robust adversarial RL and μ synthesis, and develop a model-free version of the well-known DK-iteration for solving state-feedback μ synthesis with static D-scaling. In the proposed algorithm, the K step mimics the classical central path algorithm via incorporating a recently-developed double-loop adversarial RL method as a subroutine, and the D step is based on model-free finite difference approximation. Extensive numerical study is also presented to demonstrate the utility of our proposed model-free algorithm. Our study sheds new light on the connections between adversarial RL and robust control.