Distributed Hierarchical Optimized Control for CPSs Under DoS Attacks and Mismatched Disturbances via Reinforcement Learning

• Published in: IEEE transactions on emerging topics in computational intelligence pp. 1 - 12
• Main Authors: Liu, Yang, Xie, Xiangpeng, Guo, Xiang-Gui, Sun, Jiayue
• Format: Journal Article
• Language: English
• Published: IEEE 01.01.2025
• Subjects: Artificial neural networks; Consensus control; Cyber-physical systems (CPSs); Cyberattack; denial-of-service (DoS) attacks; Denial-of-service attack; disturbance observer; hierarchical framework; Optimal control; Physical layer; Reinforcement learning; reinforcement learning (RL); Security; System performance; Vectors
• ISSN: 2471-285X; 2471-285X
• DOI: 10.1109/TETCI.2025.3592256

This article proposes an optimized consensus control based on reinforcement learning (RL) for distributed nonlinear cyber-physical systems (CPSs) subject to denial-of-service (DoS) attacks and mismatched disturbances. Through the communication among CPSs, the adverse effects of DoS attacks and disturbances are spread, which pose a significant challenge to the stability and reliability of each subsystem. Moreover, due to the DoS attacks, the differentiability of consensus error can not be guaranteed, such that the traditional control design method based on backstepping for nonlinear CPSs is not feasibility. In order to address the aforementioned issue, a hierarchical design framework is constructed comprising of a cyber layer and a physical layer. In the cyber layer, virtual nodes are constructed to facilitate information interaction, thus avoiding direct interaction in the physical layer and preventing the propagation of the impact of DoS attacks and disturbances among CPSs. In the physical layer, a high-order disturbance observer (HODO) is designed to counteract the effects of mismatched disturbances. By utilizing the reference signals generated by the virtual nodes in cyber layer to design the value functions, neural networks (NNs) approximation-based RL is performed for achieving the optimized tracking control for each subsystem in physical layer. Finally, the simulation results are provided to verify the effectiveness of our proposed control method.