stochastic dynamic programming

One of the Principles of water resources management is the optimal use of the reservoirs as the main sources of surface water, and this issue has a special importance in the science of water engineering. In this research, the new K-means clustering method to discretize reservoir inflow has been presented for the Stochastic Dynamic Programming(SDP). In addition, the Moran's method is used to discretize the reservoir storage. By the programming in the Python environment, the historical reservoir inflow in each season is classified to different clusters and obtained the best inflow cluster for each season. The effects of this clustering is also considering in the SDP of Jamishan reservoir. In general, the change in inflow classification will lead to a fundamental change in the transition probability matrix. Thus, the use of K-means method for the reservoir inflow discretization, due to the possibility of optimizing the number of clusters in each time period, can be very useful for the SDP. finally, it is strongly recommended to use K-means method to discretize reservoir inflow for reservoir operation by SDP.

K-means is an object-based algorithm that selects representative clusters from the data itself rather than averaging them. K-means of a cluster is the most central element of a cluster. The purpose of this method is to reduce sensitivity to large values in the data set. Each cluster is introduced with one of the data close to the centers. According to the number of data categories (k), the value of the least squares function is minimized and the data are categorized in the best way. In addition, the Moran's method is used to discretize the reservoir storage. In this method, the upper and lower limit of the range of changes and the upper limit of each category are used as indicators of discretization of the reservoir volume. The study area includes Jamishan reservoir sub-basin with an area of 527.07 km2 located in the southwest of Sanghar city near the Pirsalman hydrometric station. This watershed is the part of Kermanshah province which is between 32-34° to 34-53° North latitude and 47-22° to 47-52° East longitude. The annual average of rainfall, evaporation and temperature are 441 mm, 1534 mm and 10 degrees Celsius, respectively.

Evaluating the performance of the K-means model in 4 different seasons, showed that among the 19 considered clusters, the best result in seasonalclassification is obtained by the 5 inflow clusters according to the performance rate in fall, winter, spring and summer seasons - 142.57, -176.90,-475.36 and -2.10, respectively. The results of the First-order Markov chain, the possible values are given in Table 1 for 4 seasons in 5 clusters,and in this table, the specified numbers indicate the probability of moving each cluster for each season.In thefollowing, using the backward recursive function, the calculations are continued until reaching the stationary state condition. Finally, the value of l* was obtained for all 4 periods and different combinations of k and i as Fig.1. The results of Steady-state condition showed that l* happened mostly in spring up to 5 clusters of the reservoir storage and the least happened in summer with onecluster. Then, the calculations of the reservoir release probability in each period for each class of inflow and storage have been made. The highest value has occurred for reservoir storage class 4 in the autumn, winter, and spring seasons but in the summer season, due to less inflow and high water demand, it has happened in reservoir storage class 5.

In this research, the Stochastic Dynamic Programming (SDP) of the Jamishan dam reservoir is discussed using the K-means method in classifying the inflow discharge seasonally for the 41 years of historical data. Moran's method is also used to classify the storage volume of the reservoir into 7 classes. To calculate the transition probability matrix during the first-order Markov chain process, it is necessary to have the flow class in each period. For this purpose, the k-means method is used. The reservoir inflow in each season is classified from 2 to 20 classes by programming in the Python environment and especially with the Scikit-learn library. Evaluating the performance of the K-means model in 4 different seasons, showed that among the 19 considered clusters, the best result in seasonal classification is obtained by the 5 inflow clusters. Changing the number of inflow clusters leads to changes in the transition probability matrix and this process would change the results of reservoir operation. It can be said that the use of this flow classification method can have a significant impact on the management and optimization of dam reservoir performance. In general, the use of new classification methods such as the K-means method in the discretization of reservoir inflow for the reservoir stochastic dynamic programming can be very beneficial and effective.

 To deal with different water conditions, a model was developed to determine the optimal irrigation level and the optimal cropping area for major agricultural crops in lands of Moghan irrigation and drainage network in downstream of Aras Dam. In this model, in the stochastic conditions, with considering the uncertainty of required water supply, decision variables (optimal irrigation levels and optimal planting area in certain time steps) are obtained using stochastic dynamic programming (SDP) method. Expected value of the maximum profit from planting crops is considered as the return function. Also in deterministic conditions, the model was run by considering 4 scenarios. The results show the superiority of cropping pattern and optimal irrigation levels (even in water deficiency conditions) in terms of all studied factors compared to the common cropping pattern (even without water deficiency) in the region. Minimum value for factor of the required water per hectare is related to scenario 2 (cropping of low water requirement crops with pressurized irrigation in deterministic conditions) which is equal to 5368.14 m3. This factor in scenarios 1, 3 and 4 is 9079.78; 13496.25and 9211.73 m3 respectively and in the common cropping pattern in the region is 10900m3. Maximum value for factor of profit per hectare is related to scenario 2, equal to 102 million Rials. The mentioned factor for scenarios 1, 3 and 4 are 96.5, 73 and 89.5 million Rials, respectively.

Nowadays, water scarcity is the current issue in Iran. This issue made the more necessity of using the proper water resources management more than the past. Stochastic Dynamic programming (SDP) is one of the methods to obtain the reservoir operation rules. In this method, one of the most important factors to find the optimal solution is discretization of the storage capacity and reservoir inflow. In this research, some storage classes (3, 5, 7 and 10) are analyzed to achieve the optimum storage discretization by SDP method, considering tree types of objective function (α = 0, α = 0.5, α = 1) with the constant reservoir inflow classes.

In this study, the SDP model has been used to find the optimal storage of Jamishan reservoir by any objective functions. By using historical reservoir inflow time series, reservoir inflow and storage are discretized in 3 classes with equal length intervals method and also 3, 5, 7 and 10 classes by Moran method, respectively. This method is applied by driving objective function as a minimization of system damage for each composition of the reservoir inflow and storage classes (k, i). By achieving the steady policy at each period, the amount of reservoir Inflow, storage and release are deterministically defined.

The results showed that the optimal storage capacity, only water supply of downstream demands considered as an objective function, is k=7 and there is minimum water deficit in case of α=0. In addition, this would be 10 classes in case of α=1, which the amount of difference between reservoir storage and its desirable would be changed from constant value and the first decreasing change would be appear. Obtaining reservoir storage classes is also affected by method of discretization since this value is obtained 10 for classic and Moran method and 7 in Savarenskiy method. That is selected k = 10 based on the objective function in case of α = 0.5 considered two objectives of reservoir release storage volume simultaneously.

In case of α=0, the objective function is only reservoir release and water allocation, and of the optimal class of reservoir storage would occur at the point where water deficit is constant by increasing the number of storage classifications which k=7 is the optimal class. In the second scenario the objective function which is α=1 is selected as the best discretized class of the reservoir storage which has the closest vicinity to the target storage (Ts). So, in this case, k=10 is the optimum reservoir storage. In the third scenario, α = 0.5, there is a difference between to find the optimal solution when consider the TS or Tr as the criteria. Both objective function are well regarded in this case and also the first decreasing changes is happened in k=10.