Proposing a New Method for Acquiring Skills in Reinforcement Learning with the Help of Graph Clustering

Reinforcement learning is atype of machine learning methods in which the agent uses its transactions with the environment to recognize the environment and to improve its behavior.One of the main problems of standard reinforcement learning algorithms like Q-learning is that they are not able to solve large scale problems in a reasonable time. Acquiring skills helps to decompose the problem to a set of sub-problems and to solve it with hierarchical methods. In spite of the promising results of using skills in hierarchical reinforcement learning, it has been shown in some previous studies that based on the imposed task, the effect of skills on learning performance can be quite positive. On the contrary, if they are not properly selected, they can increase the complexity of problem-solving. Hence, one of the weaknesses of previous methods proposed for automatically acquiring skills is the lack of a systematic evaluation method for each acquired skill. In this paper, we propose new methods based on graph clustering for subgoal extraction and acquisition of skills. Also, we present new criteria for evaluating skills, with the help of which, inappropriate skills for solving the problem are eliminated. Using these methods in a number of experimental environments shows a significant increase in learning speed.