Out-of-the-box parameter control for evolutionary and swarm-based algorithms with distributed reinforcement learning

• Published in: Swarm intelligence Vol. 17; no. 3; pp. 173 - 217
• Main Authors: de Lacerda, Marcelo Gomes Pereira, de Lima Neto, Fernando Buarque, Ludermir, Teresa Bernarda, Kuchen, Herbert
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.09.2023; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Communications Engineering; Computer Communication Networks; Computer Science; Computer Systems Organization and Communication Networks; Evolution; Learning; Mathematical and Computational Engineering; Networks; Reinforcement; Swarm intelligence; Evolutionary algorithms; Parameter control; Reinforcement learning
• ISSN: 1935-3812; 1935-3820
• DOI: 10.1007/s11721-022-00222-z

Parameter control methods for metaheuristics with reinforcement learning put forward so far usually present the following shortcomings: (1) Their training processes are usually highly time-consuming and they are not able to benefit from parallel or distributed platforms; (2) they are usually sensitive to their hyperparameters, which means that the quality of the final results is heavily dependent on their values; (3) and limited benchmarks have been used to assess their generality. This paper addresses these issues by proposing a methodology for training out-of-the-box parameter control policies for mono-objective non-niching evolutionary and swarm-based algorithms using distributed reinforcement learning with population-based training. The proposed methodology is suitable to be used in any mono-objective optimization problem and for any mono-objective and non-niching Evolutionary and swarm-based algorithm. The results in this paper achieved through extensive experiments show that the proposed method satisfactorily improves all the aforementioned issues, overcoming constant, random and human-designed policies in several different scenarios.