Optimal Decentralized Energy Management of Electrical and Thermal Distributed Energy Resources and Loads in Microgrids Using Reinforcement Learning

In this paper, a decentralized energy management system is presented for intelligent microgrids with the presence of distributed resources using reinforcement learning. Due to the unpredictable nature of renewable energy resources, the variability of load consumption, and the nonlinear model of batteries, the design of a microgrid energy management system is associated with many challenges. In addition, centralized control structures in large-scale systems increase computational volume and complexity in control algorithms. In this paper, a fully decentralized multi-agent structure for a microgrid energy management system is proposed and the Markov decision process is used to model the stochastic behavior of agents in the microgrid. Electrical and thermal distributed resources, batteries, and consumers are considered intelligent and independent agents. They have the learning ability to explore and exploit the environment in a fully decentralized manner and achieve their optimal policies. The proposed method for hourly microgrid management is model-independent and based on learning. The method maximizes the profits of all manufacturers, minimizes consumer costs, and reduces the dependence of the microgrid on the maingrid. Finally, using real data from renewable energy sources and consumers, the accuracy of the proposed method in the Iranian electricity market is simulated and verified.