dynamic programming

This paper presents a new method for using data collected from the agent's random movement in the environment for the initial adjustment of parameters of a controller with a fuzzy reinforcement learning structure. Slow learning speed and high failure rates during training are two major challenges in such structures. The initial parameterization of the fuzzy system can be a suitable solution to address these challenges. In this paper, the method of discrete value iteration is extended to continuous without relying on derivative based methods to initialize the parameters of the fuzzy system. First, random interaction with the environment is used to collect relevant data. Since the state space is continuous, the data is appropriately clustered and each cluster is considered as a state. Then, by generalizing the standard value iteration method to the continuous, the transition probability matrix and the immediate reward expectation matrix are calculated. Using the results of this stage, the initial parameterization of the fuzzy reinforcement learning structure is performed. Subsequently, these parameters are fine-tuned using reinforcement learning. The proposed method is called "Value Iteration based Fuzzy Reinforcement Learning" and is used in the problem of target following robots. The experimental results indicate a significant improvement in the performance of the proposed method in the problem of target following robots.

Nowadays, quantum computing has played a significant role in increasing the speed of algorithms. Due to the limitations in the manufacturing technologies of quantum computers, the design of a large-scale quantum computer faces many challenges. One solution to overcome these challenges is the design of distributed quantum systems. In these systems, quantum computers are connected to each other through the teleportation protocol to transfer quantum information. Since quantum teleportation requires quantum resources, it is necessary to reduce the number of that. The purpose of this paper is to present a distributed quantum system considering the two goals of balanced distribution of qubits and minimizing the number of teleportation protocols in two levels. In the first level, by presenting a dynamic programming algorithm, an attempt has been made to distribute qubits in a balanced manner and reduce the number of connections between subsystems. According to the output partitioning obtained from the first level, in the second level and in the stage of implementation of global gates, when one of the qubits of this gate is teleported from the home to the desired destination, this qubit may be able to be used by a number of global gates, observing the precedence restrictions and as a result it reduces the number of teleportations. The obtained results show the better performance of the proposed algorithm.

Security of power supply is the energetic driver in advent and so, advance of the smart distribution networks (SDNs). The main goal of the conventional SDN expansion planning (SDNEP) is to eco-reliability supply the incremental annual demand by providing an optimal timetable for the sites and sizes of either pre-existed equipment to be expanded or new ones to be installed. Also, it gives a solution to well-operate the expanded network. SDN expansion planning (SDNEP) is to identify a cost-effective timetable for the sites and sizes of either pre-existed equipment to be replaced/reinforced or new ones to be installed in order to supply the increasing electricity load while responding to all the technical and operational constraints and requirements. In SDNs, despite the communication system plays a vital role, but, cyber-physical security of these systems exceedingly vulnerable to cyber-attacks (CAs), especially in the consumer's section. So, this paper suggests an approach for designing cyber-secure SDN (CSDN) by posing the problem as a dynamic modeling of tri-level cyber-secure SDNEP (CSDNEP) program. To evaluate the effectiveness of CSDNEP, computer simulation is done on a large scale SDN and the results are discussed.

Quantum computing is a new method of information processing that is based on the concepts of quantum mechanics and leads to strange and powerful events in the quantum field. The logic synthesis of quantum circuits refers to the process of converting a given quantum gate into a set of gates that can be implemented in quantum technologies. The most famous logic synthesis methods are CSD and QSD. The main goal of this study is to present a multi-objective logical synthesis method combining the above two methods in the quantum circuit model with the aim of optimizing the evaluation criteria. In this proposed method, the solution space is created from different combinations of CSD and QSD decomposition methods. The created solution space is a space with a very large exponential size. Then, using a bottom-up approach of multi-objective dynamic programming, a method is presented to search only a part of the entire solution space to find circuits with the optimal Pareto costs. The obtained results show that this method creates a balance between the evaluation criteria and produces many optimal Pareto solutions that can be selected according to different quantum technologies.

Today, many transportation systems use hub and spoke structures to transfer flow (good, message, passenger, etc.) from origin to destination. In such systems, the manager must plan for the location of the hubs and the allocation of other demand points to the hubs and other decisions during the planning horizon. Also, if the planning horizon is continuous time, the manager must also determine the timing of implementing the decisions. To determine the optimal decisions during the planning horizon and the best time for implementing decisions (i.e., breakpoints) according to sustainability aspects. In this research, a mathematical programming model is presented for a sustainable multi-period hub location problem in which the transportation demand between different origin-destination pairs is time-dependent and the planning horizon is continuous-time. The problem is formulated as a nonlinear multi-objective mixed integer programming model. Sustainability aspects are considered as objectives of the model. These objectives are minimizing transportation system costs, minimizing emissions in the transportation network, and maximizing fixed and variable job opportunities created by hubs during the planning horizon. Also, some valid inequalities are presented for strengthening the formulation of the problem. To solve the problem, we first use the Augmented Epsilon Constraint method version 2 (AUGMECON2) and then, use a dynamic programming approach to determine the optimal values of the breakpoints of the planning horizon. Using the proposed dynamic programming method, in each stage, some of decision variables are fixed and the number of variables in the original problem is reduced and instead of a nonlinear mixed integer programming problem, we solve a mixed integer linear programming problem that is easier to solve. The results of the solution methods are presented for a sample problem on the Turkish network dataset. Also, the CAB dataset is used to validate the dynamic programming method. The results show that the dynamic programming approach can solve problems with up to 25 nodes and 6 time periods.

In this paper, an offline energy management system (EMS) is proposed for parallel hybrid electric vehicles (HEVs). The proper energy management system is necessary for dividing torque between electrical motor and Internal Combustion Engine (ICE). The battery is a crucial component of hybrid electric vehicles and affects significantly the cost and the performance of the whole vehicle. The primary factors accelerating battery aging are high temperatures and high states of charge (SOC) of the battery. SOC is the most important state variable in EMS, and usually considered as the only dynamic variable in past researches, but the battery temperature is often considered to be constant for simplicity and the effects of EMS on the temperature variations are neglected. In this paper, first, dynamic programming is applied to a parallel HEV without considering variation of the temperature of the battery. Then, the model of battery is improved by modelling the cooling system to take into account temperature variations and show how neglecting thermal dynamics of the battery in EMS is impractical. Finally, by integrating the battery temperature as a state variable in the optimization problem, a new energy management strategy controlling variations of the battery temperature and SOC is proposed. The simulation results on tested vehicle show that in the proposed method charge and temperature of the battery is controlled so that the proposed EMS method prevents uncontrolled variations of the battery temperature and reduces the degradation rate of it.

Dynamic lot sizing problem is one of the significant problems in industrial units and has beenconsidered by many researchers. Considering the quantity discount in purchasing cost is one of the important and practical aspects of inventory problems in the field of inventory control and management, and it has been less focused in terms of stochastic version of dynamic lot-sizing problem in inventory management. In this paper, the stochastic dynamic lot-sizing problem with the assumption of existence of all-units quantity discount in purchasing is defined and formulated. Two approaches are presented to handle the solving procedure of this problem. Since the considered model is a mixed integer non-linear programing model, and the objective function of the model is the only non-linear part of the model. At first, we introduce a piecewise linear approximation model to convert the objective function to a linear term. The main solution approach breaks down the problem into four levels. At the first level, a branch and bound algorithm branches the problem on the periods with a predetermined discount level. In this case, the problem is converted to constrained version of the Sox problem [10]. This sub problem raised in each node in the branch and bound algorithm, is a mixed integer non-linear programing too, which is solved based on a dynamic programming approach in the second level. In each stage, in this dynamic programming algorithm, there is a sub -problem which is solved via a branch and bound algorithm. The problem raised in each node of the recent branch and bound algorithm is solved with lagrangian relaxation method. The numeric results found in this study indicate that the proposed approach solves the problem faster than the mathematicalprogramming model using the commercial software GAMS. Moreover, the proposed algorithm for the two discount levels is also compared with the approximate solution in the mentioned software. The results indicate that our algorithm up to 14 periods can not only obtain the exact solution, but also consume less time in contrast to the approximate model.