AoI-based Performance Optimization in RIS-Assisted mmWave Communications

With the advent of the next generation of wireless communication networks, the requirements for high-rate and timely information delivery, especially in real-time applications, have increased dramatically. In particular, Millimeter wave (mmWave) communications can provide a high data rate, and reconfigurable intelligent surfaces (RISs) can reduce the blockage sensitivity of mmWave links. Also, to quantify the data freshness, the age of information (AoI) metric is introduced in the recent literature, defined as the time elapsed since the last successfully received status update was generated. This paper aims to optimize the average AoI in a multi-user RIS-assisted millimeter wave network. For this purpose, the proposed problem was formulated as a Markov decision process (MDP) that aims to obtain the optimal control policy that minimizes the sum of the average AoI of all users. Under the unknown system statistics assumption, we propose a model-free deep reinforcement learning algorithm to schedule the users’ transmission, adjust the transmit power, and configure the RIS to reflect the signals.  The performance of the proposed method is evaluated in terms of convergence characteristics and the impact of changes in network parameters.