Neural Network-based control using Actor-Critic Reinforcement Learning and Grey Wolf Optimizer with experimental servo system validation - Alexandru Zamfirache, Iuliu; Precup, Radu-Emil; Roman, Raul-Cristian; Petriu, Emil M. (Expert systems with applications, 2023)
Flowchart diagram of the Actor-Critic RL-based control approach using GWO as the critic. [Display omitted] This paper introduces a novel reference tracking control approach implemented using a combination of the Actor-Critic Reinforcement Learning (RL) framework and the Grey Wolf Optimizer (GWO) algorithm. The classical neural network (NN)-based implementation of the Critic, optimized with the Gradient Descent (GD) algorithm, is replaced with the GWO algorithm, aiming to eliminate the main drawbacks of the GD algorithm, i.e., slow convergence and the tendency to get stuck in local optimal values. The combined effort from multiple search agents and the random values involved in the search process make the GWO algorithm very efficient in exploring the solution space and finding global optimal solutions. The main objective of the proposed approach is to build a NN-based controller capable of solving an optimal reference tracking control problem on nonlinear servo system laboratory equipment. The training data needed to build the controller is collected while the actor learns how to control the servo system, using the GWO-based critic to monitor the process and step in to correct the actor when needed. A comparison study is performed across three online RL-based control approaches, namely the novel approach using GWO to implement the Critic in the Actor-Critic RL framework, the traditional approach using NNs with GD for optimization and another approach using a metaheuristic algorithm called Particle Swarm Optimization (PSO). The experimental results illustrate the superiority of the proposed approach over the competing ones.