Q-Learning Based Supervisory PID Controller for Damping Frequency Oscillations in a Hybrid Mini/Micro-Grid

The purpose of this paper is to design a supplementary controller for traditional PID controller in order to damp the frequency oscillations in a micro-grid. Q-learning, which is used for supervise a classical PID controller in this paper, is a model free and a simple solution method of reinforcement learning (RL). RL is one of the branches of the machine learning, which is the main solution method of Markov decision process (MDPs). The proposed control mechanism is consisting of two main parts. The first part is a classical PID controller which is fixed tuned using Salp swarm algorithm. The second part is a Q‑learning based control strategy which is consistent and updates its characteristics according to the changes in the system continuously. Eventually, a hybrid micro-grid is considered to evaluate the performance of the suggested control method compared to classical PID and fractional order fuzzy PID (FOFPID) controllers. The considered hybrid system is consisting of renewable energy resources such as solar-thermal power station (STPS) and wind turbine generation (WTG), along with several energy storage devices such as batteries, flywheel and ultra-capacitor with physical constraints and time delays. Simulations are carried out in various realistic scenarios considering system parameter variations along with changing in operating conditions. Results indicate that the proposed control strategy has an excellent dynamic response compared to the traditional PID and FOFPID controllers for damping the frequency oscillations in different operating conditions.