AGENT-BASED SIMULATION OF IRANIAN ELECTRICITY MARKET BASED ON RISK AVERSE LEARNING AGENTS USING REINFORCEMENT LEARNING AND CONDITIONAL VALUE AT

Restructuring and deregulation are one of the most significant developments in the world electricity market. In this market, Generation Companies (GenCos) in an oligopolistic game with incomplete information participate in a sealed auction and offer their bids in a certain period according to the market demand. Choosing the best bid to maximize profits and minimize risks in dynamic competition with other players is one of the most important issues for GenCos. The dynamic nature of this problem can help GenCos make the best decision based on learning from the past. Using reinforcement learning and considering risk aversity of the GenCos, this paper provides an agent-based simulation of the bidding behavior of Iran's electricity market. In this simulation, the GenCos optimize their bids using a learning process based on previous bids. Although a few studies have been conducted on the modeling of risk-averse behavior of GenCos under learning conditions, risk-averse analysis based on a history of profits and losses, or sever losses, has not been focused. Thus, in this paper, the learning behavior of the GenCos is modeled by the Q-learning reinforcement learning algorithm and their risk aversion behavior is modeled by the conditional value at risk measure and risk of missed opportunities in terms of the number of auction failures (missed auction opportunities). To validate the functionality of the proposed approach, it was applied to the real data of the electricity market of Yazd province, including five GenCos with the total nominal power of 2550 MW. The results were compared for different learning conditions, risk behaviors of companies, and pay as bid and uniform pricing. The results demonstrate that learning all GenCos leads to increased competition and promoted social welfare. Also, the level of risk aversion of GenCos and the type of clearing mechanism have a direct effect on the GenCos profitability and social welfare. The results can help power plants determine the bidding strategy in competitive conditions by considering their risk level. Likewise, these results assist regulators in designing market rules in line with the actual behavior of GenCos.