Decentralized Adaptive Optimal Tracking Control for Massive Autonomous Vehicle Systems With Heterogeneous Dynamics: A Stackelberg Game

• Published in: IEEE transaction on neural networks and learning systems Vol. 32; no. 12; pp. 5654 - 5663
• Main Authors: Zhou, Zejian, Xu, Hao
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.12.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive control; Algorithms; Approximate dynamic programming (ADP); Autonomous vehicles; Control methods; Cost function; Decentralized control; Feedback control; Game theory; Games; Industrial applications; Internet of Things; mean-field games (MFGs); Multiagent systems; Neural networks; Optimal control; Probability density function; Probability density functions; reinforcement learning; Stackelberg game; Structural hierarchy; System dynamics; Tracking control; Trajectory
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2021.3100417

In this article, a decentralized optimal tracking control problem has been studied for a large-scale autonomous vehicle system with heterogeneous system dynamics. Due to the ultralarge number of agents, the notorious "curse of dimension" problem as well as the unrealistic assumption of the existence of reliable very large-scale communication links in uncertain environments have challenged the traditional multiagent system (MAS) algorithms for decades. The emerging mean-field game (MFG) theory has recently been widely adopted to generate a decentralized control method that deals with those challenges by encoding the large scale MASs' information into a novel time-varying probability density functions (PDF) which can be obtained locally. However, the traditional MFG methods assume all agents are homogeneous, which is unrealistic in practical industrial applications, e.g., Internet of Things (IoTs), and so on. Therefore, a novel mean-field Stackelberg game (MFSG) is formulated based on the Stackelberg game, where all the agents have been classified as two different categories where one major leader's decision dominates the other minor agents. Moreover, a hierarchical structure that treats all minor agents as a mean-field group is developed to tackle the assumption of homogeneous agents. Then, the actor-actor-critic-critic-mass (<inline-formula> <tex-math notation="LaTeX">A^{2}C^{2}M </tex-math></inline-formula>) algorithm with five neural networks is designed to learn the optimal policies by solving the MFSG. The Lyapunov theory is utilized to prove the convergence of <inline-formula> <tex-math notation="LaTeX">A^{2}C^{2}M </tex-math></inline-formula> neural networks and the closed-loop system's stability. Finally, a series of numerical simulations are conducted to demonstrate the effectiveness of the developed method.