Q-learning-based hyper-heuristic framework for estimating the energy consumption of electric buses for public transport

• Published in: Iran Journal of Computer Science (Online) Vol. 7; no. 3; pp. 423 - 483
• Main Authors: Turgut, Oguz Emrah, Turgut, Mert Sinan, Önçağ, Ali Çaglar, Eliiyi, Uğur, Eliiyi, Deniz Türsel
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.09.2024; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Benchmarks; Buses (vehicles); Computer Science; Computer Systems Organization and Communication Networks; Data Structures and Information Theory; Design engineering; Energy consumption; Estimation; Heuristic methods; Machine learning; Mathematics of Computing; Optimization; Public transportation; Regression models; Software Engineering/Programming and Operating Systems; Theory of Computation; Aquila optimizer; Q-learning; Regression models; Electric buses; Hyper-heuristics; Barnacles mating optimizer
• ISSN: 2520-8438; 2520-8446
• DOI: 10.1007/s42044-024-00179-8

This research study introduces a Q-learning enhanced hyper-heuristic framework for the accurate estimation of energy consumption rates of electric buses. Fundamentals of reinforcement learning concepts are hybridized with the integrated newly emerged metaheuristic methods of Aquila optimizer, Barnacles Mating Optimizer, Gradient-based Optimizer, Harris Hawks Optimization, and Poor and Rich Optimization algorithms to solve high-dimensional optimization problems with higher accuracy. In this context, the Q-learning algorithm is considered a high-level heuristic for administering the selection and move acceptance mechanisms, while search agents of those mentioned above low-level competitive metaheuristic algorithms meticulously explore the search space to find the optimum global point. Q-learning guides the operating hyper-heuristic in selecting the suitable low-level optimizer based on the Q-table score during iterations. An intelligent control mechanism is devised to get a reward or penalty for the actions of the low-level algorithms. The proposed method is evaluated on thirty-two optimization benchmark problems composed of unimodal and multimodal test functions. Then, each constituent algorithm and the hyper-heuristic model are applied to thirty-dimensional benchmark functions of CEC 2017 and twenty-eight test instances of CEC 2013. Four different challenging, complex real-world engineering design cases are also solved to assess the predictability of the proposed method on constrained problems. Finally, the proposed hyper-heuristic is employed to derive the fuel consumption estimates of electric buses. It is seen that the Multiple linear regression model, whose unknown parameters are extracted by the hyper-heuristic framework, gives the best predictions.