dynamic programming

This paper presents a new method for using data collected from the agent's random movement in the environment for the initial adjustment of parameters of a controller with a fuzzy reinforcement learning structure. Slow learning speed and high failure rates during training are two major challenges in such structures. The initial parameterization of the fuzzy system can be a suitable solution to address these challenges. In this paper, the method of discrete value iteration is extended to continuous without relying on derivative based methods to initialize the parameters of the fuzzy system. First, random interaction with the environment is used to collect relevant data. Since the state space is continuous, the data is appropriately clustered and each cluster is considered as a state. Then, by generalizing the standard value iteration method to the continuous, the transition probability matrix and the immediate reward expectation matrix are calculated. Using the results of this stage, the initial parameterization of the fuzzy reinforcement learning structure is performed. Subsequently, these parameters are fine-tuned using reinforcement learning. The proposed method is called "Value Iteration based Fuzzy Reinforcement Learning" and is used in the problem of target following robots. The experimental results indicate a significant improvement in the performance of the proposed method in the problem of target following robots.

Nowadays, quantum computing has played a significant role in increasing the speed of algorithms. Due to the limitations in the manufacturing technologies of quantum computers, the design of a large-scale quantum computer faces many challenges. One solution to overcome these challenges is the design of distributed quantum systems. In these systems, quantum computers are connected to each other through the teleportation protocol to transfer quantum information. Since quantum teleportation requires quantum resources, it is necessary to reduce the number of that. The purpose of this paper is to present a distributed quantum system considering the two goals of balanced distribution of qubits and minimizing the number of teleportation protocols in two levels. In the first level, by presenting a dynamic programming algorithm, an attempt has been made to distribute qubits in a balanced manner and reduce the number of connections between subsystems. According to the output partitioning obtained from the first level, in the second level and in the stage of implementation of global gates, when one of the qubits of this gate is teleported from the home to the desired destination, this qubit may be able to be used by a number of global gates, observing the precedence restrictions and as a result it reduces the number of teleportations. The obtained results show the better performance of the proposed algorithm.

Security of power supply is the energetic driver in advent and so, advance of the smart distribution networks (SDNs). The main goal of the conventional SDN expansion planning (SDNEP) is to eco-reliability supply the incremental annual demand by providing an optimal timetable for the sites and sizes of either pre-existed equipment to be expanded or new ones to be installed. Also, it gives a solution to well-operate the expanded network. SDN expansion planning (SDNEP) is to identify a cost-effective timetable for the sites and sizes of either pre-existed equipment to be replaced/reinforced or new ones to be installed in order to supply the increasing electricity load while responding to all the technical and operational constraints and requirements. In SDNs, despite the communication system plays a vital role, but, cyber-physical security of these systems exceedingly vulnerable to cyber-attacks (CAs), especially in the consumer's section. So, this paper suggests an approach for designing cyber-secure SDN (CSDN) by posing the problem as a dynamic modeling of tri-level cyber-secure SDNEP (CSDNEP) program. To evaluate the effectiveness of CSDNEP, computer simulation is done on a large scale SDN and the results are discussed.

Quantum computing is a new method of information processing that is based on the concepts of quantum mechanics and leads to strange and powerful events in the quantum field. The logic synthesis of quantum circuits refers to the process of converting a given quantum gate into a set of gates that can be implemented in quantum technologies. The most famous logic synthesis methods are CSD and QSD. The main goal of this study is to present a multi-objective logical synthesis method combining the above two methods in the quantum circuit model with the aim of optimizing the evaluation criteria. In this proposed method, the solution space is created from different combinations of CSD and QSD decomposition methods. The created solution space is a space with a very large exponential size. Then, using a bottom-up approach of multi-objective dynamic programming, a method is presented to search only a part of the entire solution space to find circuits with the optimal Pareto costs. The obtained results show that this method creates a balance between the evaluation criteria and produces many optimal Pareto solutions that can be selected according to different quantum technologies.

In this paper, an offline energy management system (EMS) is proposed for parallel hybrid electric vehicles (HEVs). The proper energy management system is necessary for dividing torque between electrical motor and Internal Combustion Engine (ICE). The battery is a crucial component of hybrid electric vehicles and affects significantly the cost and the performance of the whole vehicle. The primary factors accelerating battery aging are high temperatures and high states of charge (SOC) of the battery. SOC is the most important state variable in EMS, and usually considered as the only dynamic variable in past researches, but the battery temperature is often considered to be constant for simplicity and the effects of EMS on the temperature variations are neglected. In this paper, first, dynamic programming is applied to a parallel HEV without considering variation of the temperature of the battery. Then, the model of battery is improved by modelling the cooling system to take into account temperature variations and show how neglecting thermal dynamics of the battery in EMS is impractical. Finally, by integrating the battery temperature as a state variable in the optimization problem, a new energy management strategy controlling variations of the battery temperature and SOC is proposed. The simulation results on tested vehicle show that in the proposed method charge and temperature of the battery is controlled so that the proposed EMS method prevents uncontrolled variations of the battery temperature and reduces the degradation rate of it.

Revenue Management (RM) is a subfield of Operations Research that aims at maximizing revenues acquired by selling products/services in a specified period to the right customers. In this paper we address the problem of optimal assignment of general surgery ward of operation room to different classes of surgeries, each of which requiring different surgery time, using revenue management mechanisms. Deciding about accepting or rejecting a surgery request is made beside the option of preserving capacity of the operation room for future valuable requests based on the adjusted revenue (via inclusion of importance levels derived from expert opinions by an AHP approach). Data in this study were collected from Sayad-e-Shirazi Hospital of GorGan city. For formulating the problem we develop contingency and deterministic dynamic programming algorithms. Besides, we also propose a heuristic method based on  a deterministic linear programming approach. The number of accepted requests for classes with higher financial priority using RM approach are compared with FCFS policy and results signify that a higher level of efficiency is attainable using the proposed RM approaches. The efficiency along with using different surgical importance levels in accepting requests is also investigated.

In this paper, a load restoration framework is proposed in distribution network, which includes distributed generations and energy storage resources. In this framework, dynamic programming method (DP) has been used as a tool to optimize the distribution system restoration problem. The time and sequence of the feeders (loads) of the distribution network are considered as the optimal steps and the optimal answer to the problem of optimization of the dynamic programming method after the global outage has occurred. The proposed method considers the priority of network loads according to its importance in load restoration problem. At the end, the proposed method is implemented in a real network and its efficient and practical inference is derived from the results.

In this paper، design of an optimal control strategy for the powertrain of a parallel hydraulic hybrid bus is proposed. The powertrain includes an internal combustion engine as the first power generation source and a hydraulic pump/motor as the second one. Design procedure of a proper control strategy for the hybrid powertrains is extremely dependent on the speed trend of the driving cycle. This functionality is such that the control decision is affected by the future trend of the driving cycle speed، too. Here، a dynamic programming algorithm is used for generating the optimal control strategy in a special driving cycle. The disadvantage of the designed control strategy is that it is fully dependent to the future information of the driving cycle. This problem would be eliminated by using an intelligent control strategy. The control strategies including an identification unit for the driving cycle are named as the intelligent. An appropriate method to design the intelligent control strategy is using the online models of the optimal control strategies for some standard driving cycles in different time periods of a special driving cycle. In this paper، a set of models contains several neural networks is applied to generate the online models of the pre-developed optimal control strategies. The generated models are used as the online optimal control strategies on the hydraulic hybrid bus. Finally، the results of the bus simulation using the online optimal control strategy and a rule-based one are compared for assessment of the proposed design. It can be seen that the fuel consumption of the bus is reduced by using the online optimal control strategy.