RL-based Control of Smart Base Isolation System Using Unity ML-Agents

• Published in: International journal of steel structures Vol. 24; no. 4; pp. 908 - 917
• Main Authors: Kim, Hyun-Su, Kang, Joo-Won
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01.08.2024; Springer Nature B.V; 한국강구조학회
• Subjects: Algorithms; Brain research; Civil Engineering; Comparative studies; Control algorithms; Control systems; Control theory; Dynamical systems; Earthquakes; Engineering; Engines; Finite element method; Games; Isolation systems; Machine learning; Materials Science; Physics; Programming languages; Python; Robots; School environment; Seismic engineering; Seismic isolation; Seismic response; Simulation; Soft computing; Software; Solid Mechanics; Structural engineering; Unmanned aerial vehicles; 토목공학; Smart control algorithm; Smart base isolation system; Unity ML-agents; Deep Q-network; Reinforcement learning
• ISSN: 1598-2351; 2093-6311
• DOI: 10.1007/s13296-024-00862-3

Reinforcement learning (RL) has been used in the development of various control systems presenting desirable control performances. There have been many studies examining the development of structural control algorithms using conventional methods and soft computing algorithms. However, research investigating RL-based structural control techniques in particular is still in an early stage. In RL algorithms, the agent interacts with the environment by taking the appropriate action under the specific state. In the RL-based structural control problem, the environment usually includes the structure, control system, external loads, etc., and it is generally presented by the finite element model. In the present study, the Unity game engine—which has recently come to be used in various engineering simulations because of its accurate physics calculations—was used to construct a reinforcement learning environment for structural control systems. A smart base isolation system (SBIS) that was composed of a magnetorheological damper and four friction pendulum systems was used as an example structural control system, and it was modeled using the Unity physics engine for RL environment. Among various RL algorithms, a Deep Q-Network (DQN) was used to make the control algorithm for the SBIS. The command voltage for the smart base isolation was mapped into the agent’s action. The reward of the DQN algorithm was designed to be a higher value when the agent takes a better action resulting in reduced seismic responses. Three artificial ground motions were used to train the DQN-based control algorithm, and another artificial earthquake was used to investigate the control efficiency of the trained DQN-based control algorithm. The passive-on case with the maximum damper force was used for comparative study. This study shows that the DQN-based algorithm can successfully control the SBIS. The findings show that the unity game engine can accurately present the dynamic responses of the SBIS, showing that it can be effectively used for the construction of a RL environment for structural dynamic systems.