Low Carbon Economic Energy Management Method in a Microgrid Based on Enhanced D3QN Algorithm With Mixed Penalty Function

• Published in: IEEE transactions on sustainable energy Vol. 16; no. 3; pp. 1686 - 1696
• Main Authors: Zhao, Chanjuan, Li, Yunlong, Zhang, Qian, Ren, Lina
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.07.2025; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithm design and analysis; Algorithms; Carbon; Carbon content; Carbon emissions; Deep reinforcement learning; Distributed generation; Economics; Electric vehicle charging; Electric vehicles; Electrical loads; EN-D3QN-MPF; Energy management; EV users' charging satisfaction; Genetic algorithms; Low carbon economy; low-carbon economic; microgrid energy management; Microgrids; Particle swarm optimization; Penalty function; Power system dynamics; Renewable energy sources; Temperature measurement; Turbogenerators; Wind power; Wind power generation; Wind turbines; China
• ISSN: 1949-3029; 1949-3037
• DOI: 10.1109/TSTE.2025.3528952

In this paper, an enhanced dueling double deep Q network algorithm with mixed penalty function (EN-D3QN-MPF) for microgrid energy management control is developed. First, a novel microgrid model including PV, wind turbine generator, electric storage system, electric vehicle charging station, thermostatically controlled loads, and residential price-responsive loads are proposed. Then, by combining the mixed penalty function method with D3QN reinforcement learning together, a mixed penalty function method is implemented to balance the reward weightings. Accordingly, an EN-D3QN-MPF algorithm is presented to achieve low-carbon economic and EV users' charging satisfaction operation of the microgrid. The effectiveness of the proposed method is verified by the dataset collected from eastern China in 2019. Simulation results validate that our proposed method has superior energy management performance over the genetic algorithm (GA), Particle Swarm Optimization (PSO), dueling deep Q network (dueling DQN), double DQN (DDQN), and D3QN algorithms.