Fast Genetic Algorithm for feature selection — A qualitative approximation approach

• Published in: Expert systems with applications Vol. 211; p. 118528
• Main Authors: Altarabichi, Mohammed Ghaith, Nowaczyk, Sławomir, Pashami, Sepideh, Mashhadi, Peyman Sheikholharam
• Format: Journal Article
• Language: English
• Published: 01.01.2023
• Subjects: Evolutionary computation; Feature selection; Fitness approximation; Genetic Algorithm; Meta-model; Optimization; Particle Swarm Intelligence
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2022.118528

Evolutionary Algorithms (EAs) are often challenging to apply in real-world settings since evolutionary computations involve a large number of evaluations of a typically expensive fitness function. For example, an evaluation could involve training a new machine learning model. An approximation (also known as meta-model or a surrogate) of the true function can be used in such applications to alleviate the computation cost. In this paper, we propose a two-stage surrogate-assisted evolutionary approach to address the computational issues arising from using Genetic Algorithm (GA) for feature selection in a wrapper setting for large datasets. We define “Approximation Usefulness” to capture the necessary conditions to ensure correctness of the EA computations when an approximation is used. Based on this definition, we propose a procedure to construct a lightweight qualitative meta-model by the active selection of data instances. We then use a meta-model to carry out the feature selection task. We apply this procedure to the GA-based algorithm CHC (Cross generational elitist selection, Heterogeneous recombination and Cataclysmic mutation) to create a Qualitative approXimations variant, CHC QX . We show that CHC QX converges faster to feature subset solutions of significantly higher accuracy (as compared to CHC), particularly for large datasets with over 100K instances. We also demonstrate the applicability of the thinking behind our approach more broadly to Swarm Intelligence (SI), another branch of the Evolutionary Computation (EC) paradigm with results of PSO QX , a qualitative approximation adaptation of the Particle Swarm Optimization (PSO) method. A GitHub repository with the complete implementation is available. © 2022 The Author(s)