Self adaptive logic development in self adaptive systems using online deep reinforcement learning

A self-adaptive system can modify its own structure and behavior at runtime based on its perception of the environment, of itself and of its requirements. One key element of a self-adaptive system is its self-adaptation logic that encodes when and how the system should adapt itself. When developing the adaptation logic, developers face the challenge of design time uncertainty. To define when the system should adapt, they have to anticipate all potential environment states. However, anticipating all potential environment changes is infeasible in most cases due to incomplete information at design time.Online reinforcement learning (RL) addresses design time uncertainty by learning the effectiveness of adaptation actions through interactions with the system’s environment at run time, thereby automating the development of self-adaptation logic. Online-RL for self-adaptive systems integrates the elements of RL into the MAPE-K loopExisting online RL approaches for self-adaptive systems represent learned knowledge as a value function, so exhibit two shortcomings that limit the degree of automation: they require manually fine-tuning the exploration rate and may require manually quantizing environment states to foster scalability. In this paper, use policy-based deep reinforcement learning, which are structurally quite different, to automate the aforementioned manual activities. Deep RL addresses these disadvantages by representing the learned knowledge as a neural network. learned knowledge is hidden in the neural network. The results of the experiments indicate a high convergence speed of learning.