Reinforcement learning for control of flexibility providers in a residential microgrid

• Published in: IET Smart Grid Vol. 3; no. 1; pp. 98 - 107
• Main Authors: Mbuwir, Brida V, Geysen, Davy, Spiessens, Fred, Deconinck, Geert
• Format: Journal Article
• Language: English
• Published: Durham The Institution of Engineering and Technology 01.02.2020; John Wiley & Sons, Inc; Wiley
• Subjects: Active control; Algorithms; Alternative energy sources; B0240Z Other topics in statistics; B0290F Interpolation and function approximation (numerical analysis); B8110C Power system control; B8120K Distributed power generation; B8250 Solar power stations and photovoltaic power systems; C1140Z Other topics in statistics; C1340B Multivariable control systems; C3340H Control of electric power systems; C4130 Interpolation and function approximation (numerical analysis); C6170K Knowledge engineering techniques; C7410B Power engineering computing; C7420 Control engineering computing; Case Study; control engineering computing; Controllers; Cooperation; Distributed generation; distributed power generation; Efficiency; Electric power systems; Electricity; Electricity consumption; electricity consumption patterns; Energy management; Energy storage; fitted Q‐iteration; Flexibility; FQI; heat pump; Heat pumps; iterative methods; learning (artificial intelligence); Machine learning; model-based optimal controller; model-free reinforcement learning techniques; multi-agent systems; multiagent collaborative microgrid settings; Performance evaluation; photovoltaic power systems; photovoltaic production; policy iteration; power consumption; power engineering computing; power generation control; power generation scheduling; power system planning; Renewable resources; Residential development; Residential energy; residential microgrid; RL techniques; rule-based controller; single-agent stochastic microgrid settings; Smart grid; smart grid paradigm; smart meters; stochastic processes; System dynamics; Water heaters; iterative methods; multi-agent systems; smart grid paradigm; photovoltaic production; power generation control; distributed power generation; power consumption; power generation scheduling; smart meters; heat pump; multiagent collaborative microgrid settings; learning (artificial intelligence); stochastic processes; control engineering computing; fitted Q-iteration; FQI; power system planning; model-based optimal controller; photovoltaic power systems; model-free reinforcement learning techniques; RL techniques; machine learning; power engineering computing; heat pumps; single-agent stochastic microgrid settings; policy iteration; residential microgrid; electricity consumption patterns; PI; rule-based controller
• ISSN: 2515-2947; 2515-2947
• DOI: 10.1049/iet-stg.2019.0196

The smart grid paradigm and the development of smart meters have led to the availability of large volumes of data. This data is expected to assist in power system planning/operation and the transition from passive to active electricity users. With recent advances in machine learning, this data can be used to learn system dynamics. This study explores two model-free reinforcement learning (RL) techniques – policy iteration (PI) and fitted Q-iteration (FQI) for scheduling the operation of flexibility providers – battery and heat pump in a residential microgrid. The proposed algorithms are data-driven and can be easily generalised to fit the control of any flexibility provider without requiring expert knowledge to build a detailed model of the flexibility provider and/or microgrid. The algorithms are tested in multi-agent collaborative and single-agent stochastic microgrid settings – with the uncertainty due to lack of knowledge on future electricity consumption patterns and photovoltaic production. Simulation results show that PI outperforms FQI with a 7.2% increase in photovoltaic self-consumption in the multi-agent setting and a 3.7% increase in the single-agent setting. Both RL algorithms perform better than a rule-based controller, and compete with a model-based optimal controller, and are thus, a valuable alternative to model- and rule-based controllers.